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Thoso  of  us  involvod  In  the  creation  of  the  Handbook  of  Artificial  Intelligence,  both 
wrltors  and  editors,  have  attempted  to  make  the  concepts,  methods,  tools,  and  main  results 
of  artificial  Intelligence  research  accessible  to  a broad  scientific  and  engineering  audience. 
Currently  Al  work  Is  familiar  mainly  to  Its  practicing  specialists  and  other  Interested  computer 
sclontfsts.  Vet  tho  fiofd  Is  of  growing  Interdisciplinary  Interest  and  practical  Importance. 
With  this  book  we  are  trying  to  build  bridges  thot  are  easily  crossed  by  engineers,  scientists 
in  other  fiolds,  and  our  own  computor  sclonce  colleagues. 

"v  In  tho  Handbook  we  Intend  to  cover  the  breadth  and  depth  of  Al,  presenting  general 
ovorviows  of  the  sciontlfic  Issues,  as  well  as  detailed  discussions  of  particular  techniques 
and  Important  Al  systems.  Throughout  we  have  tried  to  keep  In  mind  the  reader  who  is  not  a 
specialist  In  Al. 

As  tho  cost  of  computation  continues  to  fall,  new  areas  of  application  of  computers 
becomo  potontially  viable.  For  many  of  those  areas,  there  do  not  exist  mathematical  "cores" 
to  structure  calculations!  use  of  the  computor.  Such  areas  will  Inevitably  be  served  by 
symbolic  modols  and  symbolic  inference  techniques.  Yet  those  who  understand  symbolic 
computation  have  boon  spooking  largely  to  themselves  for  twenty  years.  We  feel  that  It  Is 
urgont  for  Al  to  "go  public"  In  the  manner  Intended  by  the  Handbook. 

Sovoral  othor  writers  hove  recognized  a need  for  more  widespread  knowledge  of  Al  and 
hovo  attempted  to  help  fill  the  vacuum.  Lay  reviews.  In  particular  Margaret  Boden's 
Artificial  Intelligence  and  Natural  Man,  have  tried  to  explain  what  Is  Important  and 
Interesting  about  Al,  and  how  research  In  Al  progresses  through  our  programs.  In  addition, 
there  oro  a fow  textbooks  '.hot  ottompt  to  present  a more  detailed  view  of  selected  areas 
of  Al,  for  tho  serious  studont  of  computer  science.  But  no  textbook  con  hope  to  describe  all 
of  tho  sub-areas,  present  brief  explanations  of  the  Important  Ideas  and  techniques,  and 
review  tho  40  or  60  most  important  Al  systems.  „ 

K 

The  Handbook  contains  several  different  types  of  articles.  Key  Al  ideas  and 
tochniquos  ore  described  In  core  articles  (e.g.  basic  concepts  In  heuristic  search,  semantic 
nots).  Important  individual  Al  programs  (e.g.  SHRDIU)  are  described  In  separate  articles  that 
indicate  among  othor  things  the  designer's  goal,  the  techniques  employed,  and  the  reasons 
why  tho  program  Is  Important.  Ovorviow  articles  discuss  the  problems  and  approaches  In 
each  major  aroa.  The  overview  articles  should  be  particularly  useful  to  those  who  seek  a 
summary  of  the  underlying  Issues  that  motivate  Al  research. 

Evontually  tho  Handbook  will  contain  approximately  two  hundred  articles.  We  hope  that 
the  appearance  of  this  matorial  will  stimulate  Interaction  and  cooperation  with  other  Al 
rosoarch  altos.  We  look  forward  to  bolng  advised  of  our  inevitable  errors  of  omission  and 
commission.  And  for  a field  as  fast  moving  as  Al,  It  is  Important  that  Its  practitioners  alert  us 
to  Important  dovolopmonts,  so  that  future  editions  will  reflect  this  new  material.  We  Intend 
that  the  Handbook  of  Artificial  Intelligence  be  a living  and  changing  reference  work. 

Tho  ortlclos  In  this  edition  of  the  Handbook  were  written  primarily  by  graduate  students 
In  Al  at  Stanford  University,  with  assistance  from  graduate  students  and  Al  professionals  at 
othor  Institutions.  We  wish  particularly  to  acknowledge  the  help  from  those  at  Rutgers 
University,  SRI  International,  Xerox  Palo  Alto  Research  Center,  MIT,  and  the  RAND 
Corporation. 

The  author  of  this  report,  which  contains  the  section  of  the  Handbook  on  Search,  Is 
Anne  Gardner.  Others  who  contributed  to  or  commented  on  earlier  versions  of  this  section  - 
Include  Bruce  Buchanan,  Low  Croary,  Jim  Davidson,  Nils  Nilsson,  Ira  Pohl,  Reid  Smith,  Mark 
Stefik,  Helen  Tognettl,  and  Dave  Wilkins. 
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Foreword 


Thoso  of  us  Involvod  In  the  creation  of  the  Handbook  of  Artificial  Intelligence,  both 
writors  and  odltors,  have  attempted  to  make  tho  concopts,  methods,  tools,  and  main  results 
of  artificial  Intolllgonco  rosoarch  accosslblo  to  a broad  scientific  and  engineering  audlonce. 
Currently  Al  work  is  familiar  mainly  to  Its  practicing  specialists  and  other  Interested  computor 
sclontlsts.  Yot  tho  field  Is  of  growing  Interdisciplinary  Interest  and  practical  Importance. 
With  this  book  wo  are  trying  to  build  bridges  that  aro  easily  crossed  by  engineers,  scientists 
In  othor  flolds,  and  our  own  computor  sclonco  colleaguos. 

In  tho  Handbook  wo  intend  to  cover  tho  broadth  and  depth  of  Al,  presenting  goneral 
ovorvlows  of  the  scientific  issuos,  as  well  as  detailed  discussions  of  particular  techniques 
and  Important  Al  systems.  Throughout  we  have  tried  to  keep  In  mind  the  reader  who  Is  not  a 
specialist  in  Al. 

As  tho  cost  of  computation  continues  to  fall,  now  areas  of  application  of  computers 
bocomo  potontiolly  viable.  For  many  of  thoso  areas,  thore  do  not  exist  mathematical  "cores" 
to  structuro  calculations!  uso  of  tho  computor.  Such  areas  will  Inevitably  bo  served  by 
symbolic  modols  and  symbolic  inference  techniques.  Yot  those  who  understand  symbolic 
computation  havo  boon  speaking  largoly  to  themselves  for  twenty  years.  We  feel  that  it  Is 
urgont  for  Al  to  "go  public"  in  tho  manner  Intended  by  the  Handbook. 

Sovoral  othor  writers  have  rocognl2ed  a noed  for  more  widespread  knowledge  of  Al  and 
havo  attempted  to  hoip  fill  tho  vacuum.  Lay  roviows.  In  particular  Margaret  Boden's 
Artificial  Intelligence  and  Natural  Man,  have  triod  to  explain  what  Is  important  and 
Interesting  about  Al,  and  how  rosoarch  In  Al  progresses  through  our  programs.  In  addition, 
thoro  aro  a fow  toxtbooks  that  attompt  to  present  a more  detailed  view  of  selectod  areas 
of  Al,  for  tho  serious  student  of  computer  science.  But  no  textbook  can  hope  to  describe  all 
of  tho  sub-oroas,  prosont  brlof  explanations  of  the  Important  Ideas  and  techniques,  and 
review  the  40  or  60  most  Important  Al  systems. 

Tho  Handbook  contains  several  different  typos  of  articles.  Key  Al  Ideas  and 
tochnlquos  ore  doscrlbod  In  coro  articles  (e.g.  basic  concepts  In  heuristic  search,  semantic 
nots).  Important  Individual  Al  programs  (e  g.  SHRDLU)  are  described  In  separate  artlclos  that 
Indicate  omong  othor  things  the  designer's  goal,  tho  techniques  employed,  and  tho  reasons 
why  tho  program  Is  Important.  Ovorvlcw  articles  discuss  the  problems  and  approaches  in 
each  major  aros.  The  ovorvlew  articles  should  be  particularly  useful  to  those  who  seek  a 
summary  of  the  underlying  issuos  that  motivate  Al  research. 


Evontually  tho  Handbook  will  contain  approximately  two  hundred  artlclea.  Wo  hope  that 
tho  appearance  of  this  matorlal  will  stimulate  Interaction  and  cooperation  with  other  Al 
rosoarch  altos.  We  look  forward  to  bolng  advised  of  our  Inevitable  errors  of  omission  and 
commission.  And  for  a field  as  fast  moving  as  Al,  It  Is  Important  that  Its  practitioners  alert  us 
to  Important  dovolopmonts.  so  that  future  editions  will  reflect  this  new  material.  We  Intend 
that  tho  Handbook  of  Artificial  Intalllgence  be  a living  and  changing  reference  work. 

Tho  artlclos  In  this  edition  of  the  Handbook  wore  written  primarily  by  graduate  students 
In  Al  at  Stanford  University,  with  assistance  from  graduate  students  and  Al  professionals  at 
othor  Institutions.  We  wish  particularly  to  acknowledge  the  help  from  those  at  Rutgers 
University,  SRI  International,  Xerox  Palo  Alto  Research  Center,  MIT,  and  the  RAND 
Corporation. 

The  author  of  this  report,  which  contains  the  section  of  tho  Handbook  on  Search,  Is 
Anne  Gardner.  Othors  who  contributed  to  or  commented  on  earlier  versions  of  this  section 
Includo  Bruce  Buchanan.  Low  Croary,  Jim  Davidson,  Nils  Nilsson,  Ira  Pohl,  Reid  Smith.  Mark 
Steflk,  Helen  Tognetti,  and  Dave  Wilkins. 
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A.  Overview 


In  Artificial  Intelligence  the  terms  problem  solving  ond  search  refer  to  a large  body  of  core 
Ideas  that  deal  with  deduction,  Inference,  planning,  commonsense  reasoning,  theorem  proving, 
and  related  processes.  Applications  of  these  general  Ideas  are  found  In  programs  for  natural 
language  understanding,  information  retrieval,  automatic  programming,  ,obot:cs,  scene 
analysis,  game  playing,  expert  systems,  and  mathematical  theorem  proving.  In  this  chapter 
we  examine  search  as  a tool  for  problem  solving  In  a more  limited  area.  Most  of  the 
examples  to  be  considered  in  detail  ere  problems  that  are  relatively  easy  to  formalize.  Some 
typical  problems  are 

— finding  the  solution  to  a puzzle; 

— finding  a proof  for  a theorem  in  logic  or  mathematics; 

— finding  the  shortest  path  connecting  a set  of  nonequidlstant  points 
(the  traveling-salesman  problem); 

— finding  a sequence  of  moves  that  will  win  a game,  or  the  best  move 
to  make  at  a given  point  in  a game;  and 

— finding  a sequence  of  transformations  that  will  solve  a symbolic 
Integration  problem. 


Organization  of  the  Chapter 

This  overview  takes  a general  look  at  search  in  problem  solving,  indicating  some 
connections  with  topics  considered  in  other  chapters.  The  articles  in  the  next  section. 
Section  B,  describe  the  problem  representations  that  form  the  basis  of  search  techniques. 
The  detailed  examples  there  of  state-space  and  problem-reduction  representations  will 
clarify  what  Is  meant  by  words  like  "search"  and  "problem  solving"  in  Al.  Readers  to  whom 
the  subject  of  search  is  new  are  encouraged  to  turn  to  those  articles  for  more  concrete 
presentations  of  the  fundamental  ideas.  Section  B also  discusses  game  trees,  which  are  a 
historically  and  conceptually  important  class  of  representations. 

Section  C,  Search  Methods,  deals  with  the  algorithms  that  use  these  various  problem 
representations.  Blind  search  algorithms,  which  treat  the  search  space  syntactically,  are 
contrasted  with  heuristic  methods,  which  use  information  about  the  nature  and  structure  of 
the  problem  domain  to  limit  the  search.  Various  search  algorithms  are  presented  in  full. 

Finally,  Section  □ reviews  some  well-known  early  programs  based  on  search.  It  also 
describes  two  programs,  STRIPS  and  ABSTRIPS,  that  introduce  the  closely  related  topic  of 
planning  In  problem  solving.  This  general  topic,  however,  is  treated  more  fully  under  Planning 


Components  of  Search  Systems 

Problem-solving  systems  can  usually  be  described  In  terms  of  three  main  components. 
The  first  of  these  Is  a database,  which  describes  both  the  current  task-domaln  situation  and 
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the  goal.  Tho  database  can  consist  of  a variety  of  different  kinds  of  data  structures 
Including  arrays,  lists,  sets  of  predicate  calculus  expressions,  property  list  structures,  and 
semantic  networks.  In  theorem  proving,  for  example,  the  current  task-domain  situation 
consists  of  assertions  representing  axioms,  lemmas,  and  theorems  already  proved;  the  goal  is 
an  assertion  representing  the  theorem  to  be  prov  . :i.  In  Information  retrieval  applications,  the 
current  situation  consists  of  a set  of  facts,  and  the  goal  Is  the  query  to  be  answered.  In 
robot  problem  solving,  a current  situation  is  a uorid  model  consisting  of  statements  describing 
the  physical  surroundings  of  the  robot,  and  the  goal  is  a description  that  is  to  be  made  true 
by  a sequence  of  robot  actions. 

The  second  component  of  problem-solving  systems  is  a sot  of  operators  that  are  used  to 
manipulate  the  database.  Some  examples  of  operators  include; 

— in  theorem  proving,  rules  of  inference  such  as  modus  ponens  and 
resolution; 

--  In  chess,  rules  for  moving  chessmen; 

--  in  symbolic  Integration,  rules  for  simplifying  the  forms  to  be 
Integrated,  such  as  Integration  by  parts  or  trigonometric 
substitution. 

Sometimes  the  set  of  operators  consists  of  only  a few  general  rules  of  inference  that 
generate  new  assertions  from  existing  ones.  Usually  it  is  more  efficient  to  use  a large 
number  of  very  specialized  operators  that  generate  new  assertions  only  from  very  specific 
existing  ones. 

The  third  component  of  a problem-solving  system  is  a control  strategy  for  deciding  what 
to  do  next--in  particular,  what  operator  to  apply  and  where  to  apply  It.  Sometimes  control  Is 
highly  centralized,  in  a separate  control  executive  that  decides  how  problem-solving 
resources  should  be  expended.  Sometimes  control  is  diffusely  spread  among  the  operators 
themselves. 

Another  aspect  of  control  strategy  is  its  effect  o^  the  contents  and  organization  of  the 
database.  In  general,  the  object  is  to  achieve  the  goal  by  applying  an  appropriate  sequence 
of  operators  to  an  Initial  task-domain  situation.  Each  application  of  an  operator  modifies  the 
situation  in  some  way.  If  several  different  operator  sequences  are  worth  considering,  the 
representation  often  maintains  data  structures  showing  the  effects  on  the  task  situation  of 
each  alternative  sequence.  Such  a representation  permits  a control  strategy  that 
Investigates  various  operator  sequences  In  parallel  or  that  alternates  attention  among  a 
number  of  sequences  that  look  relatively  promising.  This  is  the  character  of  most  of  the 
algorithms  considered  In  this  chapter;  they  assume  a database  containing  descriptions  of 
multiple  task-domain  situations  or  states  (see,  e.g  , Cl,  Blind  State-space  Search).  It  may  be. 
however,  that  the  description  of  a task-domain  situation  is  too  large  for  multiple  versions  to 
be  stored  explicitly;  In  this  case,  a backtracking  control  strategy  may  be  used  (see  Al 
Programming  Languages).  A third  possibility,  available  In  some  types  of  problems  such  as 
theorem  proving,  exists  where  the  application  of  operators  can  add  new  assertions  to  the 
description  of  the  task-domain  situation  but  never  can  require  tho  deletion  of  existing 
assertions.  In  this  case,  the  database  can  describe  a single,  incrementally  changing  task- 
domain  situation;  multiple  or  alternative  descriptions  are  unnecessary.  (See  Theorem 
Proving.) 
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Reasoning  Forward  and  Reasoning  Backward 

The  application  of  oporators  to  those  structures  in  the  database  that  describe  the 
task-domain  sltuation--to  produce  > modified  situation— is  often  called  reasoning  for  ward.  The 
object  is  to  bring  the  situation,  or  problem  state,  forward  from  its  initial  configuration  to  one 
satisfying  a goal  condition.  For  example,  an  initial  situation  might  be  the  placement  of 
chessmen  on  the  board  at  the  beginning  of  the  game;  the  desired  goal,  any  board 
configuration  that  is  a checkmato;  and  the  operators,  rules  for  the  legal  moves  in  chess. 

An  alternative  strategy,  reasoning  backward,  Involves  using  another  type  of  operator, 
which  is  applied  not  to  a current  task-domain  situation  but  to  the  goal.  The  goal  statement, 
or  problem  statement,  is  converted  to  one  or  more  subgoals  that  are  (one  hopes)  easier  to 
solve  and  whose  solutions  are  sufficient  to  solve  tne  orminal  problem.  These  subgoals  may  in 
turn  be  reduced  to  sub-subgoais.  and  so  on,  until  each  of  them  is  either  accepted  to  be  a 
trivial  problem  or  its  solution  is  accomplished  by  the  solution  of  its  subproblems.  For  example. 

given  an  Initial  goal  of  Integrating  1/(cos  x)2  dx,  and  an  operator  permitting  1/(cos  x)  to  be 
rewritten  as  (sec  x),  one  can  work  backward  toward  a restatement  of  the  goal  In  a form 

whose  solution  is  immediate;  The  integral  of  (sec  x)2  is  tan  x. 

The  former  approach  Is  said  to  us e forward  reasoning  and  to  be  da:a~drivtn  or  bottom-up. 
The  latter  uses  backward  reasoning  and  Is  goal-directed  or  top-down.  The  distinction  between 
forward  and  backward  reasoning  assumes  that  the  current  task-domain  situation  or  state  is 
distinct  from  the  goal.  If  one  chooses  to  say  that  a current  state  is  the  state  of  having  a 
particular  goal,  the  distinction  naturally  vanishes. 

Much  human  problem-solving  behavior  is  observed  to  involve  reasoning  backward,  and 
many  artificial  intelligence  programs  are  based  on  this  general  strategy.  In  addition, 
combinations  of  forward  and  backward  reasoning  are  possible.  One  Important  Al  technique 
Involving  forward  and  backward  reasoning  Is  called  means-ends  analysis ; it  Involves  comparing 
the  current  goal  with  a current  task-domain  situation  to  extract  a difference  between  them. 
This  difference  is  then  used  to  index  that  (forward)  operator  most  relevant  to  reducing  the 
difference.  If  this  especially  relevant  operator  cannot  be  immediately  applied  to  the  present 
problem  state,  subgoals  are  set  up  to  change  the  problem  state  so  that  the  relevant 
operator  can  be  applied.  After  these  subgoals  are  solved,  the  relevant  oporator  is  applied 
and  the  resulting,  modified  situation  becomes  a new  starting  point  from  which  to  solve  for  the 
original  goal.  (See  02,  GPS;  and  05,  STRIPS.) 


State  Spaces  and  Problem  Reduction 

A problem-solving  system  that  uses  forward  reasoning  and  whose  operators  each  work 
by  producing  a single  new  object--a  new  state--in  the  database  is  said  to  represent 
problems  in  a state-space  representation  (see  B1). 

A distinction  may  be  drawn  between  two  cases  of  backward  reasoning.  In  one,  each 
application  of  an  operator  to  a problem  yields  exactly  one  new  problem,  whose  sire  or 
difficulty  Is  typically  slightly  less  than  that  of  the  previous  problem.  Systems  of  this  kind  will 
also  be  referred  to,  in  this  chapter,  as  employing  state-space  representations.  Two 
Instances  of  such  representations  are  presented  later  In  the  chapte,.  One  example  is  the 
Logic  Theorist  program  (01);  the  other  Is  the  backward-reasoning  part  of  Pohl's  bidirectional 
search  (Cl  and  C3d). 
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A more  complex  kind  of  backward  reasoning  occurs  if  applying  an  operator  may  divide 
the  problem  into  a set  of  subproblems,  perhaps  each  significantly  smaller  than  the  original.  An 

example  would  be  an  operator  changing  the  problem  of  integrating  2/(x2-1)  dx  Into  the  three 
subproblems  of  Integrating  1/(x-1)dx,  integrating  -1/(x+1)dx,  and  adding  the  results.  A 
system  using  this  kind  of  backward  reasoning,  distinguished  by  the  fact  that  Its  operators 
can  change  a single  object  Into  a conjunction  of  objects,  will  be  said  to  employ  a problem- 
reduction  representation.  The  relationship  between  problem-reduction  and  state-space  ■ i 

representations  Is  examined  further  at  the  end  of  Article  B2. 

There  may  or  may  not  be  constraints  on  the  order  in  which  the  subproblems  generated 
by  a problem-reduction  system  can  be  solved.  Suppose,  for  example,  that  the  original 
problem  Is  to  Integrate  (f(x)  ♦ g(x))  dx.  Applying  the  obvious  operator  changes  it  to  the  new 
problem  consisting  of  two  integrations,  f(x)  dx  and  g(x)  dx.  Depending  on  the 

representation,  the  new  problem  can  be  viewed  as  made  up  of  either  (a)  two  integration 
subproblems  that  can  be  solved  In  any  order,  or  (b)  two  integration  subproblems  plus  the 
third  subproblem  of  summing  the  Integrals.  In  the  latter  case,  the  third  task  cannot  be  done 
until  the  first  two  have  been  completed. 

Besides  the  state-space  and  problem-reduction  representation  approaches,  other 
variations  on  problem  representation  are  possible.  One  occurs  in  connection  with  game- 
playing  problems,  which  differ  from  most  other  problems  by  virtue  of  the  existence  of 
adversary  moves.  A game-playing  problem  must  be  represented  so  os  to  take  into  account 
the  opponent’s  possible  moves  as  well  as  the  player's  own.  The  usual  representation  is  a 
game  tree  (see  B3),  which  shares  many  features  of  a problem-reduction  representation. 

Another  variation  is  relevant  to  theorem-proving  systems,  many  of  which  use  forward 
reasoning  and  operators  (rules  of  inference)  that  act  on  conjunctions  of  objects  in  the 
database.  Although  the  representations  discussed  here  assume  that  each  operator  takes 
only  a single  object  os  Input,  it  is  possible  to  define  a theorem-proving  representation  that 
provides  for  multiple-input,  single-output  operators  (Kowalski,  1972;  see  Theorem  Proving). 


Graph  Representation 

In  either  a state-space  or  a problem-reduction  representation,  achieving  the  desired 
goal  can  be  equated  with  finding  an  appropriate  finite  sequence  of  applications  of  available 
operators.  While  what  one  Is  primarily  interested  In— the  goal  situation  or  the  sequence  that 
leads  to  it--may  depend  on  the  problem,  the  term  search  con  always  be  understood,  without 
misleading  consequences,  as  referring  to  the  search  for  an  appropriate  operator  sequence. 

Tree  structures  are  commonly  used  in  implementing  control  strategies  for  the  search. 
In  a state-space  representation,  a tree  may  be  used  to  represent  the  set  of  problem  states 
produced  by  operator  applications.  In  such  a representation,  the  root  node  of  the  tree 
represents  the  Initial  problem  situation  or  state.  Each  of  the  new  states  that  can  be 
producod  from  this  initial  state  by  the  application  of  just  one  operator  is  represented  by  a 
successor  node  of  the  root  node.  Subsequent  operator  applications  produce  successors  of 
these  nodes,  etc.  Each  operator  application  is  represented  by  a directed  arc  of  the  tree.  In 
general,  the  states  are  represented  by  a graph  rather  than  by  a tree  since  there  may  exist 
different  paths  from  the  root  to  any  given  node.  Trees  are  an  important  special  case, 
however,  and  it  Is  usually  easier  to  explain  tholr  use  than  that  of  graphs.  (See  B1,  State- 
space  Representation.) 
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In  addition  to  these  ordinary  trees  and  graphs  used  for  state-space  representations, 
specialized  ones  called  ANDIOR  graphs  are  used  for  problem-reduction  problem-solving 
methods.  For  problems  in  which  the  goal  can  be  reduced  to  sets  of  subgoals,  AND/OR  graphs 
provide  a means  for  keeping  track  of  which  subgoals  have  been  attempted  and  of  which 
combinations  of  subgoals  are  sufficient  to  achieve  the  original  goal  (see  Article  B2). 


The  Search  Space 

The  problem  of  producing  a state  that  satisfies  a goal  condition  can  now  be  formulated 
as  the  problem  of  searching  a graph  to  find  a node  whose  associated  state  description 
satisfies  the  goal.  Similarly,  search  based  on  a problem-reduction  representation  can  be 
formulated  as  the  search  of  an  AND/OR  graph. 

It  should  be  noted  that  there  is  a distinction  between  the  graph  to  be  searched  and 
the  tree  or  graph  that  is  constructed  as  the  search  proceeds.  In  the  latter,  nodes  and  arcs 
can  be  represented  by  explicit  data  structures;  tho  only  nodes  included  are  those  for  which 
paths  from  the  Initial  state  have  actually  boen  discovered.  This  explicit  graph,  which  grows 
as  the  search  proceeds,  will  be  referred  to  as  a search  graph  or  search  tree. 

In  contrast,  tho  graph  to  be  searched  is  ordinarily  not  explicit.  It  may  be  thought  of  as 
having  one  node  for  every  state  to  which  there  exists  a path  from  the  root.  It  may  even  be 
thought  of,  less  commonly,  as  having  one  node  for  every  state  that  can  be  described, 
whether  or  not  a path  to  It  exists.  Tho  implicit  graph  will  be  called  the  state  space  or,  if 
generalized  to  cover  non-state-space  representations  such  as  AND/OR  graphs  or  game 
trees,  tho  search  space.  Clearly,  many  problem  domains  (such  as  theorem  proving)  have  an 
infinite  search  space,  and  the  search  space  in  others,  though  finite,  Is  unimaginably  large. 
Estimates  of  search  space  size  may  bo  based  on  the  total  number  of  nodes  (however 
defined)  or  on  other  measures.  In  chess,  for  example,  the  number  cf  different  complete  plays 

of  the  average-length  game  has  boen  estimated  at  10120  (Shannon,  1960,  1956),  although 
the  number  of  "good"  games  is  much  smaller  (see  Good,  1 968).  Even  for  checkers  the  size 

of  the  search  space  nas  been  estimated  at  lO*10  (Samuel,  1963). 

Searching  now  becomes  a problem  of  making  just  enough  of  the  search  space  explicit 
in  a search  graph  to  contain  a solution  of  the  original  goal.  If  the  search  space  Is  a general 
graph,  the  search  graph  may  be  either  a subgraph,  or  a subgraph  that  Is  also  a tree,  or  a 
tree  obtained  by  representing  distinct  paths  to  one  search  space  node  with  duplicate  search 
graph  nodes. 


Limiting  Search 

The  critical  problem  of  search  is  the  amount  of  time  and  space  necessary  to  find  a 
solution.  As  the  chess  and  checkers  estimates  suggest,  exhaustive  search  Is  rarely  feasible 
for  nontrivial  problems.  Examining  all  sequences  of  n moves,  for  example,  would  require 
operating  In  a search  space  in  which  the  number  of  nodes  grows  exponentially  with  n.  Such 
a phenomenon  is  called  a combinatorial  explosion. 

There  are  several  complementary  approaches  to  reducing  the  number  of  nodes  that  a 
search  must  examine.  One  Important  way  Is  to  recast  the  problem  so  as  to  reduce  the  size  of 
the  search  space.  A dramatic,  if  well-known,  example  is  the  mutilated  chessboard  problem: 
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Suppose  two  diagonally  opposite  corner  squares  are  removed  from  a 
standard  8 by  8 square  chessboard.  Can  31  rectangular  dominoes, 
each  the  size  of  exactly  two  squares,  be  so  placed  as  io  cover 
precisely  the  remaining  board?  (Raphael,  1976,  p.  31) 

If  states  are  defined  to  be  configurations  of  dominoes  on  the  mutilated  board,  and  an 
operator  has  the  effect  of  placing  a domino,  the  search  space  for  this  problem  Is  very  large. 
If,  however,  one  observes  that  every  domino  placed  must  cover  both  a red  square  and  a 
black  one  and  that  the  squares  removed  are  both  of  one  color,  the  answer  is  immediate. 
Unfortunately,  little  theory  exists  about  how  to  find  good  problem  representations.  Some  of 
the  sorts  of  things  such  a theory  would  need  to  take  into  account  are  explored  by  Amarel 
(1968),  who  gives  a sequence  of  six  representations  for  a single  problem,  each  reducing  the 
search  space  size  by  redefining  the  states  and  operators. 

A second  aspect  concerns  search  efficiency  within  a given  search  space.  Several 
graph-  and  tree-searching  methods  have  been  developed,  and  these  play  an  important  role  in 
the  control  of  problem-solving  processes.  Of  special  interest  are  those  graph-searching 
methods  that  use  heuristic  knowledge  from  the  problem  domain  to  help  focus  the  search.  In 
some  types  of  problems,  these  heuristic  search  techniques  can  prevent  a combinatorial 
explosion  of  possible  solutions.  Heuristic  search  is  one  of  the  key  contributions  of  Al  to 
efficient  problem  solving.  Various  theorems  have  been  proved  about  the  properties  of  search 
techniques,  both  those  that  do  and  those  that  do  not  use  heuristic  Information.  Briefly,  it  has 
been  shown  that  certain  types  of  search  methods  are  guaranteed  to  find  optimal  solutions 
(when  such  exist).  Some  of  these  methods,  under  certain  comparisons,  have  also  been 
shown  to  find  solutions  with  a minimal  amount  of  search  effort.  Graph-  and  tree-searching 
algorithms,  with  and  without  the  use  of  heuristic  information,  are  discussed  at  length  In 
Section  C. 

A third  approach  addresses  the  question.-  Given  one  representation  of  a search 
problem,  can  a problem-solving  system  be  programmed  to  find  a better  representation 
automatically?  The  question  differs  from  that  of  the  first  app-oach  to  limiting  search  in  that 
here  it  Is  the  program,  not  the  program  designer,  that  is  asked  to  find  the  improved 
representation.  One  start  on  answering  the  question  was  made  by  the  STRIPS  program  (05). 
STRIPS  augments  Its  initial  set  of  operators  by  discovering,  generalizing,  and  remembering 
macro-operators,  composed  of  sequences  of  primitive  operators,  as  It  gains  problem-solving 
experience.  Another  Idea  was  used  in  the  ABSTRIPS  program  (06),  which  Implements  the 
Idea  of  planning,  In  the  sense  of  defining  and  solving  problems  in  a search  space  from  which 
unimportant  details  have  been  omitted.  The  details  of  the  solution  are  filled  In  (by  smaller 
searches  within  the  more  detailed  space)  only  after  a satisfactory  outline  of  a solution,  or 
plan,  has  been  found.  Planning  Is  a major  topic  itself;  for  further  discussion,  see  Planning. 


The  Meaning  of  "Heuristic"  and  "Heuristic  Search" 

Although  the  term  "heuristic"  has  long  been  a key  word  in  Al,  its  meaning  has  varied 
both  among  authors  and  over  time.  In  general,  its  usage  Is  illustrated  by  example  better  than 
by  definition,  and  several  of  the  prime  examples  are  Included  In  the  programs  of  Section  D. 
However,  a brief  review  of  the  ways  "heuristic"  and  "heuristic  search"  have  been  used  may 
provide  a useful  warning  against  taking  any  single  definition  too  seriously. 
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As  an  adjective,  the  most  frequently  quoted  dictionary  definition  for  "heuristic"  Is 
"serving  to  discover."  As  a noun,  roforring  to  an  obscure  branch  of  philosophy,  the  word 
meant  the  study  of  the  methods  and  rules  of  discovory  and  invention  (see  Polya,  1957,  p. 
112). 

When  the  term  came  Into  use  to  describe  Al  techniques,  some  writers  made  a 
distinction  between  methods  for  discovering  solutions  and  algorithms  for  producing  them. 
Thus  Newell,  Shaw,  and  Simon  stated  In  1957:  "A  process  that  may  solve  a given  problem, 
but  offers  no  guarantees  of  doing  so,  is  called  a heuristic  lor  that  problem"  (Newell,  Shaw,  & 
Simon,  1963b,  p.  1 14).  But  this  meaning  was  not  universally  accepted.  Minsky,  for  example, 
said  in  a 1 961  paper: 

The  adjective  "heuristic,"  as  used  hero  and  widely  In  the  literature,  means  related 
to  improving  problem-solving  performance ; as  a noun  it  Is  also  used  in  regard  to  any 
method  or  trick  used  to  improve  the  efficiency  of  a problem-solving  program.  . . . 

But  imperfect  methods  are  not  necessarily  heuristic,  nor  vice  versa.  Hence 
"heuristic"  should  not  be  regarded  os  opposite  to  "foolproof"!  this  has  caused 
some  confusion  In  the  literature.  (Minsky,  1963,  p.  407n.) 

Those  two  definitions  refer,  though  vaguely,  to  two  different  sets:  devices  that  improve 
efficiency  and  devices  that  are  not  guaranteed.  Feigenbaum  and  Feldman  (1963,  p.  6) 
apparently  limit  "heuristic"  to  devices  with  both  properties: 

A heuristic  (heuristic  rule,  heuristic  method)  is  a rule  of  thumb,  strategy,  trick, 
simplification,  or  any  other  kind  of  dovice  which  drastically  limits  search  for 
solutions  in  large  problem  spaces.  Heuristics  do  not  guarantee  optimal  solutions; 

In  fact,  they  do  not  guarantee  any  solution  at  all;  all  that  can  be  said  for  a useful 
heuristic  is  that  it  offers  solutions  which  are  good  enough  most  of  the  time. 

Even  this  definition,  however,  does  not  always  agree  with  common  usage,  because  it  lacks  a 
historical  dimension.  A device  originally  introduced  as  a heuristic  in  Feigenbaum  and 
Feldman's  sense  may  later  be  shown  to  guarantee  an  optimal  solution  after  ail.  When  this 
happens,  the  label  "heuristic"  may  or  may  not  be  dropped.  It  has  not  been  dropped,  for 
example,  with  respect  to  tho  A*  algorithm  (C3b).  Alpha-beta  pruning  (C5b),  on  the  other 
hand,  is  no  longer  called  a heuristic. 

It  should  be  noted  that  tho  definitions  quoted  above,  ranging  In  time  from  1957  to 
1963,  refer  to  heuristic  rules,  methods,  and  programs,  but  they  do  not  use  the  term 
"heuristic  search."  This  composite  term  appears  to  have  been  first  introduced  In  1966  in  a 
paper  by  Newell  and  Ernst,  "The  Search  for  Generality"  (see  Newell  & Simon,  1972,  p.  888). 
The  paper  presented  a framework  for  comparing  the  methods  used  In  problem-solving 
programs  up  to  that  time.  The  basic  framework,  there  called  heuristic  search,  was  the  one 
called  state-space  search  in  the  present  chapter.  Blind  search  methods  were  Included  within 
the  heuristic  search  paradigm. 

A similar  moaning  for  heuristic  soarch  appears  in  Nowell  and  Simon,  1972  (pp.  91-106). 
Again  no  contrast  Is  drawn  between  heuristic  search  and  blind  search;  rather,  houristlc 
search  is  distinguished  from  a problem-solving  mothod  called  generate-and-test.  The 
difference  between  the  two  is  that  the  latter  simply  generates  elements  of  the  search 
space  (l.e.,  states)  and  tests  each  in  turn  until  it  finds  one  satisfying  the  goal  condition; 
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whereas  in  heuristic  search  the  order  of  generation  can  depend  both  on  Information  gained  In 
previous  tests  and  on  the  characteristics  of  the  goal.  But  the  Newell  and  Simon  distinction  Is 
not  a hard  and  fast  one.  By  their  1976  Turing  Lecture,  they  seem  to  have  collapsed  the  two 
methods  into  one: 

Heuristic  Search.  A second  law  of  qualitative  structure  for  Al  Is  that  symbol 
systems  solve  problems  by  generating  potential  solutions  and  testing  them,  that 
Is,  by  searching.  (Newell  & Simon,  1976,  p.  126) 

In  the  present  chapter,  the  meaning  attached  to  "heuristic  search " stems  not  from 
Newell  and  Simon  but  from  Nilsson,  whose  1971  book  provides  the  most  detailed  and 
Influential  treatment  of  the  subject  that  has  yet  appeared.  For  Nilsson,  the  distinction 
between  heuristic  search  and  blind  search  is  the  Important  one.  Blind  search  corresponds 
approximately  to  the  systematic  generation  and  testing  of  search  space  elements,  but  It 
operates  within  a formalism  that  leaves  room  for  additional  Information  about  the  specific 
problem  domain  to  be  Introduced,  rather  than  excluding  It  by  definition.  If  such  information, 
going  beyond  that  needed  merely  to  formulate  a class  of  problems  as  search  problems,  Is  in 
fact  introduced,  it  may  be  possible  to  restrict  search  drastically.  Whether  or  not  the 
restriction  Is  foolproof,  the  search  Is  then  called  heuristic  rather  than  blind. 
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B.  Problem  Representation 
B1.  State-space  Representation 

A state-space  representation  of  a problem  uses  two  kinds  of  entitles:  states,  which  are 
data  structures  giving  "snapshots"  of  tho  condition  of  the  problem  at  each  stage  of  Its 
solution,  and  operators,  which  are  moans  for  transforming  the  problem  from  one  state  to 
another. 

A straightforward  example  of  state-space  representation  is  the  simple,  well-known 
puzzle  called  the  8-puzzle.  An  8-puzzle  is  a square  tray  containing  8 square  tiles  of  equal 
size,  numbered  1 to  8.  The  space  for  the  9th  tile  Is  vacant  (see  Figure  1 ). 


2 

1 

"T" 
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8 
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3 

Figure  1.  An  8-puzzle. 

A tile  may  be  moved  by  sliding  It  vertically  or  horizontally  Into  the  empty  square.  The  problem 
Is  to  transform  one  tile  configuration,  say  that  of  Figure  1,  into  another  given  tile 
configuration,  say  that  of  Figure  2. 
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Figure  2.  A solution  configuration  of 
the  8-puzzle. 


A state  Is  a particular  configuration  of  tiles;  each  state  might  be  represented  by  a 3 x 3 
matrix,  similar  to  Figures  1 and  2.  The  operators,  corresponding  to  possible  moves,  might  be 
defined  with  separate  operators  for  each  of  tiles  1 through  8.  However,  a more  concise 
definition  Is  made  possible  by  viewing  tho  empty  square  as  the  object  to  be  moved  and 
stating  the  operators  In  terms  of  the  movements  of  this  square.  In  this  formulation,  only  four 
operators  are  used: 

"UP"  (move  the  blank  up  one  square), 

"DOWN"  (move  the  blank  down  one  square), 

"LEFT"  (move  the  blank  left  one  square), 

"RIGHT"  (move  tho  blank  right  one  square). 

An  operator  may  be  inapplicable  in  certain  states,  as  when  it  would  move  the  blank  outside 
the  tray  of  tiles. 

The  set  of  ail  attainable  states  of  a problem  is  often  called  its  state  space.  The  8- 
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puzzle,  for  example,  has  a state  space  of  si2e  8!/2--since  there  are  81  configurations  of  the 
tiles  but  only  half  this  number  can  be  reached  from  any  given  starting  configuration.  This 
comes  to  only  181,440  possible  states.  For  comparison,  see  the  discussion  of  chess  and 
checkers  In  the  Overview  article. 

The  four  operators  defined  for  the  8-puzzle  form  a set  of  partial  functions  on  the  state 
space:  Each  operator,  if  It  applies  to  a given  state  at  all,  returns  exactly  one  new  state  as 
Its  result.  In  more  complex  problems,  however,  the  operators  often  contain  variables.  If,  for 
a particular  state  and  operator,  the  variables  can  be  instantiated  in  more  than  one  way,  then 
each  Instantiation  yields  one  new  state,  and  the  operators  of  the  problem,  If  they  are  to  be 
considered  as  defining  functions,  are  more  accurately  termed  operator  schemata. 

The  complete  specification  of  a state-space  problem  has  three  components.  One  is  a 
set  0 of  operators  or  operator  schemata.  In  addition,  one  must  define  a set  S of  one  or  more 
initial  states  and  find  a predicate  defining  a set  G of  goal  states.  A state-space  problem  Is 
then  the  triple  (S,  O,  G).  A solution  to  the  problem  Is  a finite  sequence  of  applications  of 
operators  that  changes  an  Initial  state  into  a goal  state. 

A state  space  can  be  treated  as  a directed  graph  whose  nodes  are  states  and  whose 
arcs  are  operators  transforming  one  state  to  another.  For  example,  if  state  1 is  a state  to 
which  any  of  three  operators  can  be  applied,  transforming  It  to  state  2,  3,  or  4,  then  the 
corresponding  graph  would  be  a3  In  Figure  3.  Nodes  2,  3,  and  4 are  called  the  successors  of 
node  1. 


node 

/j\a 

node  node  node 
2 3 4 

Figure  3.  Directed  arcs. 

In  graph  notation,  a solution  to  a state-space  problem  Is  a path  from  an  Initial  node  to  a goal 
node.  In  Figure  4,  one  solution  would  oe  an  application  of  operator  B twice,  followed  by 
operator  0,  to  reach  the  indicated  goal  node  or  final  state.  There  may  be  other  final  states 
and  multiple  ways  to  reach  a particular  final  state. 

Initial  state 

» 

operator  A / \ operator  B 

state  1 \ 

state  2 

operator  B / \ operator  C 
state  3 state  4 
operator  0 ^ 
final  state 


Figure  4.  A state-space  graph. 
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A common  variation  on  state-space  problems  requiros  finding  not  Just  any  path  but  one 
of  minimum  cost  between  an  Initial  node  and  a goal  node.  In  this  case,  each  arc  of  the  graph 
Is  labeled  with  Its  cost.  An  example  is  the  traveling-salesman  problem:  Given  a number  of 
cities  to  be  visited  and  the  mileage  between  each  pair  of  cities,  find  a minimum-mileage  trip 
beginning  and  ending  at  city  A that  visits  each  of  the  other  cities  exactly  once.  An  example 
mileage  chart  and  the  corresponding  state-space  graph  are  shown  In  Figure  6.  Because 
different  paths  to  the  same  city  represent  distinct  partial  solutions,  each  state  Is  identified 
not  Just  as  a city  name  but  as  a list  of  the  citios  visited  so  far. 
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Figure  5.  A traveling-salesman  problem. 


The  desired  solution  is  A-B-D-C-A,  or  its  reversal,  with  a total  mileage  of  26.  (The  two 
bottom  levels  of  the  graph  could  be  omitted,  since  the  mileage  of  each  tour  of  n cities  is 
determined  by  the  first  n-1  cities  chosen  to  be  visited.) 

Because  the  state-space  graph  Is  usually  too  large  to  represent  explicitly,  the  problem 
of  searching  for  a solution  becomes  one  of  generating  Just  enough  of  the  graph  to  contain 
the  desired  solution  path.  Search  methods  are  discussed  in  Article  Cl,  Blind  State-space 
Search,  and  Section  C3,  Heuristic  State-space  Search. 
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See  Nilsson  (1071). 
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B2.  Problem-reduction  Representation 

Often  distinguished  from  the  state-space  representation  of  problems  is  a technique 
called  problem-reduction  repre Mutation.  In  the  problem-reduction  approach,  the  principal  data 
structures  are  problem  descriptions  or  goals.  An  initial  problem  description  is  given;  it  is 
solved  by  a sequence  of  transformations  that  ultimately  change  it  Into  a set  of  subproblems 
whose  solutions  are  Immediate.  The  transformations  permitted  are  defined  as  operators.  An 
operator  may  change  a single  problem  into  several  subproblems;  to  solve  the  former,  all  the 
subproblems  must  be  solved.  In  addition,  several  different  operators  may  be  applicable  to  a 
single  problem,  or  the  same  operator  may  be  applicable  in  several  different  ways.  In  this 
case  It  suffices  to  solve  the  subproblems  produced  by  any  one  of  the  operator  applications. 

A problem  whose  solution  is  immediate  is  called  a primitive  problem.  Thus,  a problem 
representation  using  problem  reduction  is  defined  by  a triple  consisting  of 

(a)  an  Initial  problem  description, 

(b)  a set  of  operators  for  transforming  problems  to  subproblems,  and 

(c)  a set  of  primitive  problem  descriptions. 

Reasoning  proceeds  backward  from  the  initial  goal. 


An  Example 

An  example  that  lends  itself  nicely  to  problem-reduction  representation  is  the  famous 
Tower  of  Hanoi  puzzle.  In  one  common  version  there  are  three  disks,  A,  B,  and  C,  of 
graduated  sizes.  There  are  also  three  pegs,  1,  2,  and  3.  Initially  the  disks  are  stacked  on 
peg  1,  with  A,  the  smallest,  on  top  and  C,  the  largest,  at  the  bottom.  The  problem  is  to 
transfer  the  stack  to  peg  3,  as  In  Figure  1,  given  that  (a)  only  one  disk  can  be  moved  at  a 
time,  and  (b)  no  disk  may  be  placed  on  top  of  a smaller  disk. 


Initial  State  Goal  State 


peg  1 peg  2 peg  3 peg  1 peg  2 peg  3 


Figure  1.  The  Tower  of  Hanoi  puzzle. 

Only  one  operator  need  be  used  in  the  solution:  Given  distinct  pegs  i,  J,  and  k,  the 
problem  of  moving  a stack  of  size  n > 1 from  peg  I to  peg  k can  be  replaced  by  the  three 
problems: 


(a)  moving  a stack  of  size  n-1  from  i to  j, 

(b)  moving  a stack  of  size  1 from  i to  k,  and 

(c)  moving  a stack  of  size  n-1  from  J to  k. 

The  only  primitive  problem  is  that  of  moving  a single  disk  from  one  peg  to  another,  provided 
no  smaller  disk  Is  on  the  receiving  peg.  If  a smaller  disk  were  present,  this  problem  would  be 
unsolvable  (in  view  of  the  definition  of  the  only  available  operator). 
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Each  problem  description  can  now  be  given  by  specifying  the  size  n of  the  stack  to  bo 
moved,  the  number  of  the  sending  pog,  and  the  number  of  the  receiving  peg.  The  criginel 
problem,  moving  a stack  of  threo  disks  from  peg  1 to  peg  3,  would  then  be  represented  as 
(n  * 3,  1 to  3),  and  tho  transformation  of  tho  original  problem  to  primitive  problems  can  be 
represented  by  a tree: 


1 to  2 


ill 

to  3 


!i 

i)  (i 

1 to  3 


2 to  3 


(?j  (el  rb 

n*l  n=l  n=l 

1 to  3 1 to  2 3 to  2 


n*i 
2 to  1 


<< 
n=  l 
2 to  3 


U0) 

n= 1 

1 to  3 


Figure  2.  Solution  of  tho  Tower  of  Hanoi  puzzle. 


There  happen  to  be  two  possible  operator  sequences  that  transform  the  original 
problem  to  primitive  problems:  Apply  tho  operator  to  node  1,  then  node  2,  and  then  node  4; 
or  apply  the  operator  to  node  1,  then  node  4,  and  then  node  2.  Since  node  3 is  a primitive 
problem,  It  needs  no  further  attention.  Node  2 represents  the  subproblem  of  moving  the  top 
two  disks  on  peg  1 to  peg  2.  This  subproblom  is  solved  by  expanding  it  to  the  primitive 
problems  at  nodes  (6),  (6),  and  (7)--which  are  solved  by  moving  the  smallest  disk  to  peg  3, 
moving  the  middle  disk  to  peg  2,  and  finally  putting  the  small  disk  back  on  top  of  the  middle 
one. 


The  sequence  of  operators  to  be  applied  should  be  distinguished  from  the  sequence  of 
actions  to  be  taken  to  achieve  the  goal.  In  the  Tower  of  Hanoi  example,  the  actions  are  the 
actual  movements  of  the  disks.  This  sequence  is  given  by  tho  terminal  nodes  of  the  tree, 
read  left  to  right.  Whether  or  not  it  Is  considered  important  to  assemble  such  a sequence  of 
actions  depends  on  the  particular  problem  domain. 


AND/OR  Graphs 

In  tho  example  above,  a treo  was  used  to  display  a problem-reduction  solution  to  the 
Tower  of  Hanoi  puzzle.  The  tree  notation  must  be  generalized  If  It  Is  to  represent  the  full 
variety  of  situations  that  may  occur  in  problem  reduction.  This  generalized  notation  for 
problem  reduction  is  called  an  AND/OR  graph. 

According  to  one  common  formulation  (Nilsson,  1971),  an  AND/OR  graph  is  constructed 
according  to  the  following  rules: 

1.  Each  node  represents  either  a single  problem  or  a set  of  problems  to  be 
solved.  The  graph  contains  a start  node  corresponding  to  the  original  problem. 
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2.  A node  representing  a primitive  problem,  called  a Itrninal  ncnit , has  no 
descendants. 

3.  For  oach  possible  application  ot  an  operator  to  problem  P.  transforming  it  to  a 
set  of  subproblems,  there  is  a directed  arc  from  P to  a node  representing  the 
resulting  subproblem  set.  For  example,  Figure  3 illustrates  the  reduction  of  P 
to  three  different  subproblem  sots:  A,  B,  and  C.  Since  P can  be  solved  if  any 
one  of  sets  A,  B,  or  C can  be  solved,  A,  B,  and  C are  called  OR  ru\its. 


P 


Figure  3.  An  AND/OR  tree. 


4.  Figure  3 illustrates  further  the  composition  of  sets  A,  B,  and  C:  A « (D,  E).  B 
consists  of  a single  (unnamed)  problem,  and  C * (F,  G,  H).  In  general,  for  each 
node  representing  a sot  of  two  or  more  subprobloms,  there  are  directed  arcs 
from  the  node  for  the  set  to  individual  nodes  for  each  subproblem.  Since  a set 
of  subprobloms  can  be  solved  only  if  its  members  can  all  be  solved,  the 
subproblem  nodes  are  callod  AND  ru\its.  To  distinguish  thorn  from  OH  nodes, 
the  arcs  leading  to  AND  node  successors  of  a common  parent  are  Joined  by  a 
horizontal  line. 

6.  A simplification  of  the  graph  producod  by  rules  3 and  4 may  be  made  In  the 
special  case  where  only  one  application  of  an  operator  is  possible  for  problem 
P and  where  this  operator  produces  a set  of  more  than  one  subproblem.  As 
Figure  4 illustrates,  the  mum/diait  OR  nocit  representing  the  subproblem  set 
may  then  be  omitted: 


Figure  4.  An  AND/OR  tree  with  one 
oporator  at  problem  P. 

Another  example  of  this  construction  was  givon  In  Figure  2. 

In  the  figures  above,  every  node  represents  a distinct  problem  or  set  of  problems. 
Since  each  node  except  the  start  node  has  just  one  parent,  the  graphs  are  in  fact  ANDIOR 
trtts.  As  a variation  on  Figure  3.  assume  that  problem  A is  reducible  to  D and  E:  and  problem 
C,  to  E,  G,  and  H.  Then  E may  be  represented  either  by  two  distinct  nodes,  or  by  a single 
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node  as  shown  In  Figure  6.  The  choice  makes  a difference  in  the  search  algorithms  which 
are  discussed  later  in  the  chapter.  For  example,  if  node  E is  in  turn  reducible  to  C,  the 
general  graph  representation  simply  adds  another  directed  arc  to  Figure  6,  but  the 
corresponding  tree  becomes  infinite. 

P 


a 

E 

Figure  5.  An  AND/OR  graph. 


The  constructions  discussed  so  far  concern  graphs  depicting  the  entire  problem  search 
space.  To  find  a solution  to  the  initial  problem,  one  need  only  build  enough  of  the  graph  to 
demonstrate  that  the  start  node  can  be  solved.  Such  a subgraph  is  called  a solution  graph  or, 
in  the  more  restricted  case  of  an  AND/OR  tree,  a solution  tree.  The  following  rules  apply: 

A node  is  solvable  if: 

(a)  It  Is  a terminal  node  (a  primitive  problem); 

(b)  It  Is  a nonterminal  node  whose  successors  are  AND  nodes  that  are 
all  solvable;  or 

(c)  It  is  a nonterminal  node  whose  successors  are  OR  nodes  and  at 
least  one  of  them  is  solvable. 

Similarly,  a node  Is  unsolvable  If: 

(a)  It  is  a nonterminal  node  that  has  no  successors  (a  nonprimitive 
problem  to  which  no  operator  applies); 

(b)  it  is  a nonterminal  node  whoso  successors  are  AND  nodes  and  at 
least  one  of  them  is  unsolvable;  or 

(c)  It  Is  a nonterminal  node  whose  successors  are  OR  nodes  and  all  of 
them  are  unsolvable. 

Methods  of  searching  an  AND/OR  graph  for  such  a solution  are  discussed  in  Articles  C2  and 
C4. 


. 


Relation  between  Problem-reduction  and  State-space  Representations 

Some  Interesting  general  relationships  can  be  found  between  problem-reduction  and 
state-space  representations.  In  the  first  place,  although  one  representation  often  seems 
the  more  natural  for  a given  problem,  it  is  often  possible  to  recast  the  problem  definition  so 
that  It  uses  the  other  form.  For  example,  the  Tower  of  Hanoi  puzzle  can  also  be  solved  by  a 
state-space  search  using  operators  that  move  a single  disk  and  that  represent  all  the  legal 
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moves  in  a given  configuration.  In  comparison  to  the  problem-reduction  representation,  which 
In  fact  gives  an  algorithm  for  solving  the  puzzle,  the  state-space  representation  would  be  a 
poor  one  since  It  leaves  room  for  searching  down  unnecessarily  long  paths. 

Socond,  It  Is  possible  to  translate  mechanically  betweon  state-space  representations 
and  problem-reduction  representations  without  any  fundamental  shift  in  the  way  a problem  is 
viewed.  The  ways  of  making  such  translations  can  provide  helpful  insight  Into  many  search 
programs  In  which  the  concepts  of  state-space  and  problem-reduction  representation  appear 
to  be  Intermixed.  Several  translation  schemes  are  described  below.  (Some  readers  may 
wish  to  skip  the  following  material  at  first  reading.) 

State  space  to  problem  reduction.  Two  approaches  suggest  themselves  for 
translating  state-space  representations  to  problem-reduction  representations.  In  one,  the 
state-space  graph  Is  understood  as  an  AND/OR  graph  containing  only  OR  nodes.  Each  state 
of  the  state-space  version  corresponds  to  the  problem  of  getting  from  that  state  to  a goal 
state;  and  a goal  state  of  the  state  space  becomes  the  primitive  problem  of  getting  from 
that  goal  state  to  itself.  In  other  words,  data  structures  representing  states  are  simply 
reinterpreted  as  representing  problem  descriptions,  where  a problem  consists  of  state 
Information  together  with  an  implicit  goal. 


Alternately,  there  is  a slight  variation  of  the  first  approach  that  requires  redefining  the 
operators  of  the  state-space  representation.  Each  such  operator,  taking  state  I to  state  j, 
becomes  an  operator  applicable  to  the  problem  of  getting  from  state  I to  a goal  state.  Its 
effect  is  to  reduce  the  problem  to  a pair  of  subproblems:  (a)  go  from  state  I to  state  j (a 
primitive  problem),  and  (b)  go  from  state  J to  a goal  state.  Figure  6 Illustrates  this 
correspondence. 


State  f 

State  J 


(6a) 


Go  from  state  t to  goal  state 


Go  from  state  i 
to  state  j 
(a  primitive  problem] 


Go  from  state  j 
to  goal  state 


(6b) 


Figure  6.  (a)  Part  of  a state-space  tree;  (b)  the  corresponding 
part  of  an  AND/OR  (problem-reduction)  tree. 


Problem  reduction  to  state  space.  The  translation  from  a problem-reduction 
representation  to  a state-space  representation  is  a littie  more  complex,  assuming  that  the 
problem-reduction  operators  in  fact  produce  AND  nodes.  The  initial  problem  of  the  problem- 
reduction  representation  can  be  understood  as  having  two  components:  (a)  the  description 
of  the  goal  to  be  achieved,  as  discussed  at  the  beginning  of  this  article,  and  (b)  the 
description  of  an  Initial  state  of  the  world.  These  components  will  be  denoted  gQ  and  §5, 
respectively.  Some  examples  are 

**  flQ  ■ * theorem  to  be  proved,  and  sO  = the  axioms  from  which  to  prove  it; 

”■  flO  ■ a configuration  of  objects  to  be  achieved,  and  5O  * their  existing  configuration. 
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Each  state  S of  the  corresponding  state-space  representation  is  a pair  consisting  of  a stack 

of  goals  (gl gO)  to  be  achieved  and  a current  state  s of  the  world.  Thus,  the  Initial  state 

SO  of  the  state-space  representation  Is  SO*  ((gO),  sO).  A final  state  Is  one  In  which  the 
stack  ofrgoals  to  be  achieved  has  been  emptied. 


For  oach  problem-reduction  operator,  mapping  a problem  or  goal  g to  a set  of  subgoals 
{gm,  ...,  gn),  the  state-space  representation  has  a corresponding  operator  mapping  state  SI, 

where  SI  ■ ((gl,  ....  gO),  s),  to  a state  S2  in  which  (gm gn)  have  been  added  to  th«»  top 

of  the  goal-stack  (in  the  order  in  which  they  should  be  carried  out,  if  relevant),  and  the  ate 
of  the  world  s Is  unchanged;  that  is,  S2  = ((gm gn,  gi,  ....  gO),  s). 


The  state-space  representation  also  needs  a second  type  of  operator,  which  becomes 
applicable  whenever  the  goal  on  top  of  the  stack  represents  a primitive  problem.  Its  function 
is  to  remove  that  primitive  problem  from  the  stack  and,  at  the  same  time,  to  change  the  state 
s to  reflect  Its  solution.  In  the  Tower  of  Hanoi  puzzle,  for  example,  the  new  state  would 
reflect  the  changed  position  of  a single  disk.  In  a theorem-proving  problem,  the  new  state 
would  differ  from  the  old  one  by  the  addition  of  one  formula  to  those  that  had  been  given  as 
axioms  or  established  from  having  solved  previous  subproblems.  A representation  of  this 
type  is  used  explicitly  In  Fikes  and  Nilsson's  STRIPS  program,  described  in  Article  05. 
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B3.  Game  Trees 

Most  games  played  by  computer  programs,  including  checkers,  chess,  go,  and  tic-tac- 
too,  hove  several  basic  features  in  common.  There  are  two  players  who  alternate  in  making 
moves.  At  each  turn,  the  rules  define  both  what  moves  are  legal  and  the  effect  that  each 
possible  move  will  have;  thore  Is  no  dement  of  chance.  In  contrast  to  card  games  In  which 
the  players*  hands  are  hidden,  each  player  has  complete  Information  about  his  opponent's 
position,  Including  the  choices  open  to  him  and  the  moves  he  has  made.  The  game  begins 
from  a specified  state,  often  a configuration  of  men  on  a board.  It  ends  In  a win  for  one 
player  and  a loss  for  the  othor,  or  possibly  in  a draw. 

A complete  game  tree  is  a representation  of  all  possible  plays  of  such  a game.  The  root 
nodo  Is  the  Initial  state,  in  which  It  is  the  first  player's  turn  to  movo.  Its  successors  are  the 
states  he  can  reach  in  one  move;  their  successors  are  tho  states  resulting  from  the  other 
player's  possible  replies;  and  so  on.  Terminal  states  are  those  representing  a win,  loss,  or 
draw.  Each  path  from  the  root  node  to  a terminal  node  gives  a different  complete  plry  of  the 
game. 


An  Important  difference  between  a game  tree  and  a state-space  tree  (Article  Bl)  is 
that  the  game  tree  represents  moves  of  two  opposing  players,  say  A and  B.  An  AND/OR  tree 
(Article  B2),  however,  is  sufficient  to  reflect  this  opposition.  The  game  tree  is  ordinarily 
drawn  to  represent  only  one  player's  point  of  view.  In  a game  treo  drawn  from  A's 
standpoint,  A's  possible  moves  from  a given  position  are  represented  by  OR  nodes  since  they 
are  alternatives  under  his  own  control.  The  moves  that  B might  make  in  return  are  AND 
nodes,  since  they  represent  sets  of  moves  to  which  A must  be  able  to  respond.  Because  the 
players  take  turns,  OR  nodos  and  AND  nodos  appear  at  alternate  levels  of  the  tree.  In  the 
language  of  AND/OR  graphs,  the  tree  displays  the  search  space  for  the  problem  of  showing 
that  A can  win.  A node  representing  a win  for  A corresponds  to  a primitive  problem;  a node 
representing  a win  for  B or  a draw,  to  an  unsolvable  problem.  Unlike  the  usual  AND/OR  graph 
terminology,  both  of  those  kinds  of  nodes  will  bo  called  terminal  nodes. 

As  an  example,  Figure  1 shows  a portion  of  the  game  treo  for  tic-tac-toe.  The  players 
are  X and  O,  X has  the  first  movo,  and  the  treo  is  drawn  from  X's  standpoint.  Positions  are 
considered  Identical  if  one  can  be  obtained  from  the  other  by  rotation  or  reflection  of  the 
grid.  Tho  treo  could  also  be  drawn  from  0‘s  standpoint,  ovon  though  X has  the  first  move.  In 
this  case,  the  AND  nodes  would  become  OR  nodes,  and  vice  versa,  and  the  labels  "win"  and 
"lose"  would  be  reversed.  An  alternate  formulation  of  game  trees,  not  explicitly 
distinguishing  between  AND  and  OR  nodos,  is  given  in  Article  C5o,  Minimax. 

Methods  of  searching  a game  treo  for  a winning  strategy  are  discussed  in  Section  C5. 
As  with  search  In  other  domains,  tho  source  of  difficulty  in  challenging  games  Is  the 
unimaginably  large  search  space.  A complete  gamo  treo  for  checkers,  for  instance,  which  is 
harder  than  tic-tac-toe  but  far  simpler  than  chess  or  go,  has  been  estimated  as  having  about 

1 040  nonterminal  nodes  (Samuel,  1063).  If  one  assumed  that  these  nodes  could  be 
generated  at  the  rate  of  3 billion  per  second,  generation  of  the  whole  tree  would  still  require 

around  102’  centuriosl 
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C.  Search  Methods 

Cl.  Blind  State-space  Search 

As  discussed  In  Article  B1,  a problem  in  the  state-space  search  paradigm  is  defined  by 
a triple  (S,  O,  G),  where 


S Is  a set  of  one  or  more  initial  states. 

0 is  a sot  of  operators  on  states,  and 
G Is  a set  of  goal  states. 

The  state  space  Is  commonly  identified  with  a directed  graph  in  which  each  node  Is  a state 
,and  each  arc  represents  the  application  of  an  operator  transforming  a state  to  a successor 
state.  A solution  is  a path  from  a start  state  to  a goal  state.  Goal  states  may  be  defined 
either  explicitly  or  as  the  set  of  states  satisfying  a given  predicate. 

The  search  for  a solution  is  conducted  by  making  Just  enough  of  the  state-space  graph 
explicit  to  contain  a solution  path.  If  the  order  In  which  potential  solution  paths  are 
considered  is  arbitrary,  using  no  domain-specific  Information  to  judge  where  the  solution  is 
hkely  to  lie,  the  search  Is  called  blind  starch.  Although  blind  search  is  Impracticable  for 
nontrivial  problems,  because  of  the  large  proportion  of  the  state  space  it  may  explore,  it 
provides  a useful  foundation  for  the  understanding  of  heuristic  starch  techniques,  discussed  in 
Section  C3. 

Several  blind-search  methods  are  described  below;  they  differ  from  one  another  mainly 
In  the  order  In  which  nodes  aro  axaminod.  In  each  case,  It  is  assumed  that  a procedure 
exists  for  finding  all  the  successors  of  a given  nodo--that  Is,  all  the  states  that  can  be 
reached  from  the  current  state  by  a single  operator  application.  Such  a procedure  Is  said  to 
expand  the  given  node. 

The  first  three  algorithms  also  make  two  other  assumptions: 

(a)  The  state-space  graph  Is  a tree.  The  Implication  is  that  there  Is  only  one 
start  state  (the  root)  and  that  the  path  from  the  start  node  to  any  other 
node  Is  unique.  Modifications  to  the  search  methods  needed  for  a general 
directed  graph  are  noted  in  Nilsson  (1071)  and  In  Article  C3a,  Basic 
Concepts  in  Heuristic  Search. 

(b)  Whenever  a node  Is  expanded,  creating  a node  for- each  of  its  successors, 
the  successor  nodes  contain  pointers  back  to  the  parent  node.  When  a goal 
node  is  finally  generated,  this  feature  makes  it  possible  to  trace  the  solution 
path. 


Breadth-first  Search 

The  breadth-first  method  expands  nodes  in  order  of  their  proximity  to  the  start  node, 
measured  by  the  number  of  arcs  botwoen  them.  In  other  words,  It  considers  every  possible 
operator  sequence  of  length  n before  any  sequence  of  length  n+1.  Thus,  although  the 
search  may  be  an  extremely  long  one,  It  Is  guaranteed  eventually  to  find  the  shortest 
possible  solution  sequence  If  any  solution  exists. 
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Breadth-first  search  Is  described  by  the  following  algorithm: 

(1)  Put  the  start  node  on  a list,  called  OPEN,  of  unexpanded  nodes.  If  the 
start  node  Is  a goal  node,  the  solution  has  been  found. 

(2)  If  OPEN  is  empty,  no  solution  exists. 

(3)  Remove  the  first  nodo,  n,  from  OPEN  and  place  It  in  a list,  called  CLOSED, 
of  expanded  nodes. 

(4)  Expand  node  n.  If  It  has  no  successors,  go  to  (2). 

(6)  Place  all  successors  of  node  n at  the  end  of  the  OPEN  list. 

(6)  If  any  of  tho  successors  of  node  n Is  a goal  node,  a solution  has  been 
found.  Otherwise,  go  to  (2). 

» 

As  an  example  of  broadth-first  search,  consider  a world  consisting  of  a table  and  three  >. 

toy  blocks.  The  initial  state  of  the  world  is  that  blocks  2 and  3 are  on  the  table,  and  block  1 
is  on  top  of  block  2 (see  Figure  1).  We  wish  to  reach  a goal  state  in  which  the  three  blocks  . 

are  stacked  with  block  1 on  top,  block  2 in  the  middle,  and  block  3 on  the  bottom.  < 


! 

i 

i 


Initial  state 


Goal  state 


Figure  1.  An  examplo  problem  for  breadth-first  search. 

The  only  operator  is  MOVE  X to  Y,  which  moves  object  X onto  another  object,  Y.  As 
preconditions  to  applying  tho  operator,  it  is  required  (a)  that  X,  the  object  to  be  moved,  be  a 
block  with  nothing  on  top  of  it,  and  (b)  that  if  Y is  a block,  there  must  be  nothing  on  Y, 
Finally,  the  operator  Is  not  to  be  usod  to  generate  the  same  state  more  than  once.  (This  last 
condition  can  be  checked  from  the  lists  of  expanded  and  unexpanded  nodes.) 

Figure  2 shows  tho  search  tree  generated  by  tho  breadth-first  algorithm.  The  nodes 
are  states  SO  through  S10;  node  SI.  for  examplo,  corresponds  to  the  successor  state  of  SO 
reached  by  "MOVE  block  1 to  the  table."  The  nodes  are  generated  and  expanded  in  the 
order  given  by  their  state  numbers,  1.  e„  SO,  51,  S2,  ...  , S10.  When  the  algorithm  terminates, 
finding  SI 0 to  be  the  goal  state,  the  list  of  expanded  nodes  contains  SO  through  S5,  and  the 
OPEN  list  still  contains  S6  through  S10. 
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SO: 


51: 

1 2 3 


S4 : 2 S5:  2 S6:  3 

13  13  12 

S9:  3 Sit:  1 

2 2 

1 3 

Figure  2.  The  search  tree  for  Figure  1. 


Uniform-cost  Search 

The  breadth-first  algorithm  can  be  generalized  slightly  to  solve  the  problem  of  finding 
the  cheapest  path  from  the  start  state  to  a goal  state.  A nonnegative  cost  is  associated 
with  every  arc  joining  two  nodes;  the  cost  of  a solution  path  is  then  the  sum  of  the  arc  costs 
along  the  path.  The  generalized  algorithm  Is  called  a uniform-cost  search,  if  all  arcs  have 
equal  cost,  the  algorithm  reduces  to  broadth-first  search.  The  need  for  assigning  costs  to 
the  arcs  Is  illustrated  by  the  traveling-salesman  problem,  described  In  Article  Bl,  where  the 
different  distances  between  cities  correspond  to  the  arc  costs  and  the  problem  is  to 
minimize  the  total  distance  traveled. 

In  the  uniform-cost  algorithm  given  below,  the  cost  of  the  arc  from  node  I to  node  J Is 
denoted  by  c(i,J).  The  cost  of  a path  from  the  start  node  to  any  node  I Is  denoted  g(l). 

(1)  Put  the  start  node,  s,  on  a list  called  OPEN  of  unexpanded  nodes.  If  the 
start  node  is  a goal  node,  a solution  has  been  found.  Otherwise,  set 
g(s)  = 0. 

(2)  If  OPEN  Is  empty,  no  solution  exists. 

(3)  Select  from  OPEN  a node  I such  that  g(i)  is  minimum.  If  several  nodes 
qualify,  choose  node  I to  be  a goal  node  if  there  Is  one;  otherwise,  choose 
among  them  arbitrarily.  Move  node  I from  OPEN  to  a list,  CLOSED,  of 
expanded  nodes. 

(4)  If  node  I is  a goal  node,  the  solution  has  been  found. 

(6)  Expand  node  i.  If  it  has  no  successors,  go  to  (2). 

(6)  For  each  successor  node  J of  node  I,  compute  g(J)  ■ g(i)  ♦ c(I.J)  and  place 

. alt  the  successor  nodes  j in  OPEN. 

(7)  Go  to  (2). 


1 

2 3 


S7: 


3 

1 2 


S2:  1 

2 3 
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Depth-first  Search 

Depth-first  search  Is  characterized  by  the  expansion  of  the  most  recently  generated, 
or  deepest,  node  first.  Formally,  the  < itpth  of  a node  in  a tree  is  defined  as  follows: 

The  depth  of  the  start  node  is  0. 

The  depth  of  any  other  node  is  one  more  than  the  depth  of  its  predecessor. 

As  a consequence  of  expanding  the  deepest  node  first,  the  search  follows  a single  path 
through  the  state  space  downward  from  the  start  node;  only  If  It  reaches  a state  that  has  no 
successors  does  it  consider  an  alternate  path.  Alternate  paths  systematically  vary  those 
previously  tried,  changing  only  the  last  n steps  while  keeping  n as  small  as  possible. 

In  many  problems,  of  course,  the  state-space  tree  may  be  of  infinite  depth,  or  at  least 
may  be  deeper  than  some  known  upper  bound  on  the  length  of  an  acceptable  solution 
sequence.  To  prevent  consideration  of  paths  that  are  too  long,  a maximum  Is  often  placed  on 
the  depth  of  nodes  to  be  expanded,  and  any  node  at  that  depth  Is  treated  as  If  It  had  no 
successors.  It  should  be  noted  that,  even  If  such  a depth  bound  Is  used,  the  solution  path 
found  is  not  necessarily  the  shortest  one. 

The  following  algorithm  describes  depth-first  search  with  a depth  bound: 

(1)  Put  the  start  node  on  a list,  OPEN,  of  unexpended  nodes.  If  It  is  a 
goal  node,  a solution  has  been  found. 

(2)  If  OPEN  Is  empty,  no  solution  exists. 

(3)  Move  the  first  node,  n,  on  OPEN  to  a list  CLOSED  of  expanded 
nodes. 

(4)  If  the  depth  of  node  n is  equal  to  the  maximum  depth,  go  to  (2). 

(5)  Expand  node  n.  If  it  has  no  successors,  go  to  (2). 

(6)  Place  all  successors  of  node  n at  the  beginning  of  OPEN. 

(7)  If  any  of  the  successors  of  node  n is  a goal  node,  a solution  has 
been  found.  Otherwise  go  to  (2). 

As  an  example,  consider  tho  following  simple  problem:  A pawn  Is  required  to  move 
through  the  matrix  In  Figure  3 from  top  to  bottom.  The  pawn  may  enter  the  matrix  anywhere 
In  the  top  row.  From  a square  containing  0,  the  pawn  must  move  downward  If  the  square 
below  contains  0;  otherwise,  it  must  move  horizontally.  From  a square  containing  1 , no 
further  moves  are  possible.  The  goal  is  to  reach  a square  containing  zero  In  the  bottom  row. 
A depth  bound  of  6 Is  assumed. 
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Figure  3.  An  example  problem  for  depth-first  search. 
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The  search  tree  generated  by  tho  depth-first  algorithm  Is  shown  In  Figure  4.  At  node 
SO,  the  pawn  has  not  yet  entered  the  grid.  At  tho  other  nodes,  its  position  is  given  as  a (row 
number,  column  number)  pair.  The  numbering  of  nodes  gives  the  order  in  which  they  are 
moved  out  of  the  OPEN  list  of  unexpanded  nodes.  When  the  algorithm  terminates,  the  OPEN 
list  contains  SI  7 (a  goal  node)  and  SIS;  all  other  nodes  ore  on  the  expanded  list.  The 
solution  found,  which  Is  one  move  longer  than  the  minimum,  calls  for  the  pawn  to  enter  at 
(1,3),  move  one  square  right,  and  then  go  straight  down  to  (4.4).  Had  no  depth  bound  been 
used,  the  tree  would  have  been  one  level  deeper  since  node  S12  has  a successor,  (4,1). 
Since  the  algorithm  treats  the  state  space  as  a tree,  not  a general  graph,  it  does  not 
discover  that  the  distinct  nodes  S2  and  S9  in  fact  represent  the  same  state.  Consequently, 
the  search  downward  from  S9  duplicates  the  work  already  done  from  S2. 


Si 

(1.1) 


Figure  4.  The  search  tree  for  Figure  3. 
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Bidirectional  Search 

Each  of  the  algorithms  giver,  above  uses  forward  reasoning,  working  from  the  start  node 
of  a state-space  tree  towards  a goal  node  and  using  operators  that  each  map  a node  I to  a 
successor  node  j.  In  some  cases,  the  search  could  equally  well  use  backward  rtasoning, 
moving  from  the  goal  state  to  the  start  state.  An  example  of  this  Is  the  8-puzzle,  in  which 

(a)  the  goal  state  can  be  fully  described  in  advanco,  and 

(b)  It  is  easy  to  define  inverse  operators--each  applicable  operator  mapping 
node  J to  a predecessor  node  i. 

Since  backward  search  through  a tree  is  trivial,  it  Is  assumed  that  node  j can  have  more  than 
one  predecessor— that  is,  several  Inverse  operators  may  apply  at  node  J.  For  example,  in 
the  pawn  maze  problem,  Figure  4,  position  (1,2)  [at  nodes  S2  and  S9]  would  have  both  nodes 
SO  and  S8  as  predecessors. 

Forward  and  backward  reasoning  can  be  combined  into  a technique  called  bidirtctional 
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starch.  The  idea  Is  to  replace  a singlo  search  graph,  which  is  likely  to  grow  exponentially,  by 
two  smaller  graphs:  one  starting  from  the  initial  state  and  one  starting  from  the  goal.  The 
search  terminates  (roughly)  when  the  two  graphs  Intersect. 

A bidirectional  version  of  the  uniform-cost  algorithm,  guaranteed  to  find  the  shortest 
solution  path  through  a general  state-space  graph,  Is  due  to  Pohl  (1069,  1071).  Empirical 
data  for  randomly  generated  graphs  showed  that  Pohl's  algorithm  expanded  only  about  one- 
fourth  as  many  nodes  as  unidirectional  search. 

An  algorithm  to:  blind  bidirectional  search  Is  given  in  detail  below.  A related  algorithm 
for  hturistic  bidirectional  search  Is  discussed  in  Article  C3d. 

The  following  notation  is  used  in  the  algorithm; 

The  start  node  is  s;  the  goal  or  terminal  node,  t. 

S-OPEN  and  S-CLOSED  are  lists  of  unexpanded  and  expanded  nodes, 
respectively,  generated  from  the  start  node. 

T-OPEN  and  T-CLOSED  are  lists  of  unexpanded  and  expanded  nodes, 
respectively,  generated  from  the  terminal  node. 

The  cost  associated  with  the  arc  from  node  n to  node  x is  denoted  c(n,x). 

For  a node  x generated  from  the  start  node,  gs(x)  measures  the  shortest 
path  found  so  tar  trom  s to  x. 

For  a node  x generated  from  the  terminal  node,  gt(x)  measures  the  shortest 
path  found  so  far  from  x to  t. 

The  algorithm  Is  as  follows-. 

(1)  Put  s in  S-CLOSED,  with  gs(s)  - 0.  Expand  node  s,  creating  a node  for  each 
of  Its  successors.  For  each  successor  node  x,  place  x on  S-OPEN,  attach  a 
pointer  back  to  s,  and  set  gs(x)  to  c(s,x).  Correspondingly,  put  t In  T- 
CLOSED,  with  gt(t)  « 0.  Expand  node  t,  creating  a node  for  each  if  Its 
predecessors.  For  each  predecessor  node  x,  place  x on  T-OPEN,  attach  a 
pointer  forward  to  t,  and  set  gt(x)  ■ c(x,t). 

(2)  Decide  whether  to  go  forward  or  backward.  If  forward,  go  to  (3);  if 
backward,  to  (4).  (One  way  to  implement  this  step  is  to  alternate  between 
forward  and  backward  moves.  Another  way,  which  Pohl  found  to  give  better 
performance,  Is  to  move  backward  If  T-OPEN  contains  fewer  nodes  than  S- 
OPEN;  otherwise,  forward.  It  Is  assumed  that  a solution  path  does  exist,  so 
the  chosen  list  will  be  nonempty.) 

(3)  Select  trom  S-OPEN  a node  n at  whlcn  gs(n)  is  minimum.  Move  n to  S- 
CLOSED.  If  n Is  also  In  T-CLOSED,  go  to  (5).  Otherwise,  for  each  successor 
x of  n : 

(a)  If  x is  on  noithor  S-OPEN  nor  S-CLOSED,  then  add  It  to  S-OPEN. 

Attach  a pointer  back  to  n and  the  path  cost  gs(x)  ■ gs(n)  ♦ c(n,x). 
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(b)  If  x was  already  on  S-OPEN,  a shorter  path  to  x may  have  Just  been 
found.  Compare  the  previous  path  cost,  gs(x),  with  the  new  coat 
gs(n)  ♦ c(n,x).  If  the  latter  is  smaller,  set  gs(x)  to  the  new  path 
cost  and  point  x back  to  n instead  of  its  predecessor  on  the  longer 
path. 

(c)  If  x was  already  on  S-CLOSED,  do  nothing;  although  a new  path  to  x 
has  been  found,  its  cost  must  be  at  least  as  great  as  the  cost  of  the 
path  already  known.  (For  further  consideration  of  this  point,  see 
Article  C3b.) 

Return  to  (2). 

(4)  Select  from  T-OPEN  a node  n at  which  gt(n)  Is  minimum.  Move  n to  T-CLOSED. 
If  n is  also  in  S-CLOSED,  go  to  (5).  Otherwise,  for  each  predecessor  x of  n: 

(a)  If  x is  on  neither  T-OPEN  nor  T-CLOSED,  then  add  It  to  T-OPEN. 
Attach  a pointer  forward  to  n and  the  path  cost 
Qt(x)  * gt(n)  ♦ c(x,n). 

(b)  If  x was  already  on  T-OPEN  and  a shorter  path  from  x to  t has  Just 
been  found,  reduce  the  stored  value  of  gt(x),  and  point  x forward  to 
n (Instead  of  to  Its  successor  on  the  longer  path). 

(c)  If  x was  already  on  T-CLOSED,  do  nothing. 

Return  to  (2). 

(6)  Consider  the  set  of  nodes  that  are  in  both  S-CLOSED  and  either  T-CLOSED  or 
T-OPEN.  Select  from  this  set  a node  n tor  which  gs(n)  ♦ gt(n)  Is  minimum;  and 
exit  with  the  solution  path  obtained  by  tracing  the  path  from  n back  to  s and 
forward  to  t. 


References 

See  Nilsson  (1971),  Pohl  (1069),  and  Pchl  (1971). 
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C2.  Blind  AND/OR  Graph  Search 

A problem  to  be  solved  using  AND/OR-graph  search  can  be  defined  by  specifying  a 
start  node  (representing  an  Initial  goal  or  problem  description),  a set  of  terminal  nodes 
(descriptions  of  primitive  problems),  and  a set  of  operators  for  reducing  goals  to  subgoals. 

The  rules  for  constructing  an  AND/OR  graph,  together  with  the  use  of  such  graphs  for 
problem-reduction  representation,  were  discussed  in  Article  B2.  To  recapitulate  briefly,  each 
possible  application  of  an  operator  at  a node  n (see  Figure  1)  is  represented  by  a directed 
arc  from  node  n to  a successor  node;  these  successor  nodes  are  called  OR  nodes,  since  only 
one  of  the  operator  applications  will  ever  be  needed  to  solve  the  problem  that  node  n 
represents.  Each  OR  node  successor  of  node  n represents  a set  of  subproblems.  If  the  set 
of  subproblems  represented  by  an  OR  node  m has  more  than  one  element,  then  there  are 
directed  arcs  from  m to  nodes  representing  the  Individual  elements  of  the  set.  These 
successors  are  called  AND  noties,  because  aH  of  the  elements  of  the  set  must  be  solved  In 
order  to  solve  the  subproblem  set  represented  by  node  m.  To  distinguish  AND  nodes  visually 
from  OR  nodes,  the  arcs  In  the  graph  from  m to  Its  AND  successors  are  Joined  by  a horizontal  • 

line. 


Figure  1.  AND/OR  graph  notation. 


Formally,  a node  or  problem  is  said  to  be  solved  if  one  of  the  following  conditions  holds: 

1.  The  node  Is  in  the  set  of  terminal  nodes  (primitive  problems).  (In  this 
case,  the  node  has  no  successors.) 

2.  The  node  has  AND  nodes  as  successors  and  all  these  successors  are 
solved. 

3.  The  node  has  OR  nodes  as  successors  and  any  one  of  these 
successors  is  solved. 

A solution  to  the  original  problem  is  given  by  a subgraph  of  the  AND/OR  graph  sufficient  to 
show  that  the  start  node  Is  solved.  In  Figure  2,  for  example,  assuming  that  nodes  6,  6,  8,  6, 
10,  and  11  are  all  terminal,  there  are  three  possible  solution  subgraphs:  {1,  2,  4,  8,  8},  {1, 
3,  6,  6.  7,  10),  and  {1,  3,  6,  6,  7,  1 1). 
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Figure  2.  An  AND/OR  graph. 

A node  Is  said  to  be  unsolvabit  If  one  of  the  following  conditions  is  true: 


1.  The  node  has  no  successors  and  is  not  in  the  set  of  terminal  nodes. 

That  is,  It  is  a nonprimitivo  problem  to  which  no  operator  can  be 
applied. 

2.  The  node  has  AND  nodes  as  successors  and  one  or  more  of  these 
successors  is  unsoivable. 

3.  The  node  has  OR  nodes  as  successors  and  all  of  these  succesors  are 
unsoivable. 

Again  in  Figure  2,  node  1 would  be  unsoivable  if  all  nodes  in  any  of  the  following  sets  were 
unsoivable;  {8,6),  {8,6),  {8.10,11),  (9,5).  {9.6),  {9,10,11). 

Two  algorithms  for  the  blind  search  of  an  AND/OR  tree  (breadth-first  and  depth-first) 
are  given  at  the  end  of  this  article.  They  have  several  features  in  common  with  blind  state- 
space  search  algorithms  (Article  Cl);  The  operation  of  pending  a node  Is  again  present, 
and  again  the  algorithms  differ  mainly  in  the  order  in  which  nodes  are  considered  for 
expansion.  It  should  be  noted  that  the  expansion  of  a node  may  differ  slightly  from  the  case 
of  state-space  search.  In  Figure  2,  for  example,  two  operators  apply  at  node  1 : One 
reduces  it  to  a single  equivalent  problem  (node  2)  and  the  other  to  a set  (node  3)  of  three 
subproblems  (nodes  6,  6,  and  7).  In  this  case,  nodes  2,  3,  5,  6,  and  7 would  all  be  generated 
In  expanding  node  1,  and  each  new  node  would  be  given  a pointer  to  Its  Immediate 
predecessor,  but  only  nodes  2,  6,  6,  and  7 would  be  placed  on  the  list  of  unexpanded  nodes. 

In  contrast  to  the  state-space  soarch  algorithms,  most  of  which  use  forward  reasoning, 
the  search  algorithms  below  reason  backward  from  the  initial  goal.  The  algorithms  described 
here  make  two  Important  simplifying  assumptions:  (a)  The  search  space  is  an  AND/OR  tree 
and  not  a general  graph,  and  (b)  when  a problem  is  transformed  to  a set  of  subproblems,  the 
subproblems  may  be  solved  In  any  order.  The  first  assumption  implies  that  identical 
subproblems  may  arise  at  different  nodes  of  the  search  tree  and  will  need  to  be  solved  anew 
whenever  one  of  them  Is  encountered.  Modifications  needed  for  searching  a general  AND/OR 
graph  are  discussed  in  Nilsson  (1971).  A way  of  eliminating  the  second  assumption,  that  all 
subproblems  are  independent,  is  discussed  In  Article  C4,  Heuristic  Search  of  an  AND/OR 
Graph. 
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Breadth-first  Search  of  an  AND/OR  Tree 

The  following  algorithm  describes  the  breadth-first  search  of  an  AND/OR  tree.  If  a 
solution  tree  exists,  this  algorithm  finds  a solution  tree  of  minimum  depth,  provided  that 
intermediate  OR  nodes  are  ignored  in  calculating  the  depth  of  the  tree.  The  start  node  is 
assumed  not  to  be  a terminal  node. 

(1)  Put  the  start  node  on  a list,  OPEN,  of  unexpanded  nodes. 

(2)  Remove  the  first  node,  n,  from  OPEN. 

(3)  Expand  node  n--generating  all  its  immediate  successors  and,  for  each 
successor  m,  If  m represents  a set  of  more  than  one  subproblem,  generating 
successors  of  m corresponding  to  the  individual  subproblems.  Attach,  to 
each  newly  generated  node,  a pointer  back  to  its  immediate  predecessor. 

Place  all  the  new  nodes  that  do  not  yet  have  descendants  at  the  end  of 
OPEN. 

(4)  If  no  successors  were  generated  in  (3),  then 
^a)  Label  node  n unsolvable. 

(b)  If  the  unsolvability  of  n makes  any  of  its  ancestors  unsolvable,  label 
these  ancestors  unsolvable. 

(c)  If  the  start  node  Is  labeled  unsolvable,  exit  with  failure. 

(d)  Remove  from  OPEN  any  nodes  with  an  unsolvable  ancestor. 

(6)  Otherwise,  if  any  terminal  nodes  were  generated  In  (3),  then 

(a)  Label  these  terminal  nodes  solved. 

(b)  If  the  solution  of  these  terminal  nodes  makes  any  of  their  ancestors 
solved,  label  these  ancestors  solved. 

(c)  If  the  start  node  is  labeled  solved,  exit  with  success. 

(d)  Remove  from  OPEN  any  nodes  that  are  labeled  solved  or  that  have  a 
solved  ancestor. 

(6)  Go  to  step  2. 


Depth-first  Search  of  an  AND/OR  Tree 

A bounded  depth-first  search  can  be  obtained  by  changing  only  step  3 of  the  breadth- 
first  algorithm.  The  revised  step  3 is  as  follows: 

(3‘)  If  the  depth  of  n is  less  than  the  depth  bound,  then:  Expand  node  n,  generating 
all  Its  Immediate  successors  and,  for  each  successor  m,  if  m represents  a 
set  of  more  than  one  subproblem,  generating  successors  of  m corresponding 
to  the  Individual  subproblems.  Attach,  to  each  newly  generated  node,  a 
pointer  back  to  its  Immediate  predecessor.  Place  all  the  new  nodes  that  do 
not  yet  have  descendants  at  the  beginning  of  OPEN. 

The  depth-first  search  will  find  a solution  tree,  provided  one  exists  within  the  depth  bound. 
As  with  breadth-first  search,  the  notion  of  depth  is  more  meaningful  if  intermediate  OR  nodes 
are  not  counted.  For  this  purpose  one  might  add  the  following  to  the  end  of  step  3': 

For  each  node  x added  to  OPEN,  set  the  depth  of  x to  be  the  depth 
of  node  n,  plus  1 . 
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Given  that  the  etart  node  haa  depth  0,  the  depth  of  any  node 
operator  sequence  that  must  be  applied  to  reach  node  x from 


x will  then  be  the  length  of  the 
the  start  node. 


References 


See  Nilsson  (1071). 
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C3.  Heuristic  State-space  Search 


C3a.  Basic  Concepts  in  Heuristic  Search 

In  the  blind  search  of  a state-space  (Article  Cl)  or  an  AND/OR  graph  (Article  C2),  the 
number  of  nodes  expanded  before  reaching  a solution  is  likely  to  be  prohibitively  large. 
Because  the  order  of  expanding  the  nodes  is  purely  arbitrary  and  does  not  use  any 
properties  of  the  problem  being  solved,  one  usually  runs  out  of  space  or  time  (or  both)  in  any 
but  the  simplest  problems.  This  result  is  a manifestation  of  the  combinatorial  explosion. 

Information  about  the  particular  problem  domain  can  often  be  brought  to  bear  to  help 
reduce  the  search.  In  this  section,  it  is  assumed  that  the  definitions  of  initial  states, 
operators,  and  goal  states  all  are  fixed,  thus  determining  a search  space;  the  question,  then, 
is  how  to  search  the  given  space  efficiently.  The  techniques  for  doing  so  usually  require 
additional  information  about  the  properties  of  the  specific  problem  domain  beyond  that  which 
is  built  into  the  state  and  operator  definitions.  Information  of  this  sort  will  be  called  heuristic 
information,  and  a search  mothod  using  it  (whether  or  not  the  method  is  foolproof)  will  be 
called  a heuristic  search  method  (Nilsson,  1971). 


The  Importance  of  Heuristic  Search  Theory 

Heuristic  search  methods  were  employed  by  nearly  all  early  problem-solving  programs. 
Most  of  these  programs,  though,  were  written  to  solve  problems  from  a single  domain,  and  the 
domain-specific  information  they  used  was  closely  intertwined  with  the  techniques  for  using 
it.  Thus  the  heuristic  techniques  themselves  were  not  easily  accessible  for  study  and 
adaptation  to  new  problems,  and  there  was  some  likelihood  that  substantially  similar 
techniques  would  have  to  bo  reinvented  repeatedly.  Consequently,  an  Interest  arose  in 
developing  generalized  heuristic  search  algorithms,  whose  properties  could  be  studied 
independently  of  the  particular  programs  that  might  use  them.  (See  Newell  & Ernst,  1965; 
Feigenbaum,  1969;  Sandewall,  1971.)  This  task,  in  turn,  required  a way  of  describing 
problems  that  generalized  across  many  different  domains.  Such  generalized  problem 
formulations  have  been  discussed  in  Section  B,  Problem  Representation,  in  an  approach 
generally  following  Nilsson  (1971).  Given  a gei  alized  problem  representation,  the  most 
basic  heuristic  search  techniques  can  be  studied  as  variations  on  blind  search  methods  for 
the  same  type  of  problem  representation 

The  current  state  of  heuristic  search  theory  has  been  diversely  judged.  One  of  the 
best  known  students  of  the  subject  has  remarked,  "The  problem  of  efficiently  searching  a 
graph  has  essentially  been  solved  and  thus  no  longer  occupies  Al  researchers"  (Nilsson, 
1974).  Other  work  makes  it  clear,  however,  that  the  theory  is  far  from  complete  (e.g., 
Gaschnig,  1077;  Simon  & Kadane,  1975).  Its  kinship  with  corhplexity  theory  now  tends  to  be 
emphasized  (see  Pohl,  1977). 


Ways  of  Using  Heuristic  Information 

The  points  at  wh, ch  heuristic  information  car  be  applied  In  a search  Include 

(a)  deciding  which  node  to  expand  next,  instead  of  doing  the  expansions  in  a 
strictly  breadth-first  or  depth-first  order; 
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(b)  In  the  course  of  expanding  a node,  deciding  which  successor  or 
successors  to  generate--instead  of  blindly  generating  all  possible 
successors  at  one  time;  and 

(c)  deciding  that  certain  nodes  should  be  discarded,  or  pruned,  from  the 
search  tree. 


A state-space  search  algorithm  is  presented  below  that  uses  heuristic  Information  only 
at  the  first  of  these  points,  deciding  which  node  to  expand  next,  on  the  assumption  that 
nodes  are  to  be  expanded  fully  or  not  at  all.  The  general  idea  is  always  to  expand  the  node 
that  seems  "most  promising."  A search  that  implements  this  idea  is  called  an  ordered  search  or 
best-first  search.  Ordered  search  has  been  the  subject  of  considerable  theoretical  study,  and 
several  variations  on  the  basic  algorithm  below  are  reviewed  in  articles  IIC3b  through  IIC3d 
(ordered  state-space  search)  and  articlo  IIC4  (ordered  AND/OR  graph  search). 

The  other  two  uses  of  heuristic  Information  can  be  discussed  more  brio  I i » Decisions  of 
the  second  klnd--determining  which  successors  to  generate--are  often  decisions  of  operator 
selection,  determining  which  operator  to  apply  next  to  a given  node  A node  to  which  some 
but  not  all  applicable  operators  have  been  applied  Is  said  to  have  been  partially  developed  or 
partially  expanded.  The  uso  of  heuristic  Information  to  develop  nodes  partially,  reserving  the 
possibility  of  fuller  expansion  at  a later  point  in  the  search,  has  been  Investigated  by  Michie 
(1967)  and  by  Michie  and  Ross  (1970).  Other  applications  of  tho  idea  of  limiting  the 
successors  of  a given  node  occur  in  game-playing  programs  (see  C5c).  Another  important 
variant  of  the  Idea  Is  means-ends  analysis,  which,  instead  of  deciding  on  an  applicable 
operator,  chooses  an  operator  most  likely  to  advance  the  search  whether  or  not  It  Is 
Immediately  applicable.  The  problem  of  making  the  operator  applicable,  if  necessary,  .s 
addressed  secondarily.  (See  D2,  GPS;  and  05,  STRIPS.) 

The  third  use  of  heuristic  information,  for  pruning,  amounts  to  deciding  that  some  nodes 
should  never  be  expanded.  In  some  cases,  it  con  be  definitely  determined  that  a node  is  not 
part  of  a solution,  and  the  node  may  then  be  safely  discarded,  or  pruned,  from  the  search 
tree.  In  other  cases  pruning  may  be  desirable  even  though  the  nodes  pruned  cannot  be 
guaranteed  inessential  to  a solution.  One  reason,  in  conjunction  with  a best-first  search,  is 
simply  to  save  the  space  that  would  bo  required  to  retain  a large  number  of  apparently 
unpromising  nodes  on  a list  of  candidates  for  possible  future  expansion.  For  examples,  see 
Doran  (1967)  and  Harris's  bandwidth  search  (article  IIC3c).  Another  reason  for  pruning  Is  as  a 
restriction  on  a search  that  Is  otherwise  blind.  For  example,  a breadth-first  search  could  be 
modified  to  choose  between  expansion  and  pruning  for  each  node  it  considers.  This  pruning 
to  control  the  search  is  also  very  important  for  problems  in  which  all  solutions,  rather  than 
Just  a single  solution,  must  be  found;  for  finding  all  solutions  Implies  an  exhaustive 
exploration  of  all  unpruned  parts  of  tho  soarch  space.  An  example  of  a search  for  all 
solutions  Is  the  DENDRAL  program  (see  Applications.Dondrol). 


Ordered  State-space  Search 

An  ordered  or  best-first  search,  os  mentioned  above,  is  one  that  always  selects  the  most 
promising  node  as  the  next  node  to  expand.  The  choice  Is  ordinarily  assumed  to  be  global, 
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that  is,  to  operate  on  the  set  of  all  nodes  generated  but  not  yet  expanded.  A local  choice 
would  also  be  possible,  howevor;  for  example,  an  ordtrtd  dtpth-first  starch  would  be  one  that 
always  expands  the  most  promising  successor  of  the  node  last  expanded. 

The  promise  of  a node  can  bo  defined  in  various  ways.  One  way,  In  a state-space 
problem,  Is  to  estimate  Its  distance  from  a goal  node;  another  Is  to  assume  that  the  solution 
path  Includes  the  node  being  evaluated  end  estimate  the  length  or  difficulty  of  the  entire 
path.  Along  a different  dimension,  the  evaluation  may  consldor  only  certain  predetermined 
features  of  the  node  In  question,  or  it  may  determine  the  relevant  featuros  by  comparing  the 
given  node  with  the  goal.  In  all  these  cases,  the  measure  by  which  the  promise  of  a node  is 
estimated  Is  called  an  /valuation  function. 

A basic  algorithm  for  ordcrod  state-space  search  Is  given  by  Nilsson  (1071)  The 
evaluation  function  is  f";  It  is  defined  so  that  tho  more  promising  a nodo  is,  the  smaller  is  the 
value  of  f“.  The  node  solocted  for  expansion  Is  one  at  which  f*  is  minimum.  The  state  space 
Is  assumed  to  be  a general  graph. 

The  algorithm  Is  as  follows: 

(1)  Put  tho  start  nodo  s on  a list,  callod  OPEN,  of  unexpanded  nodes.  Calculate 
f*(s)  and  associate  its  value  with  node  s. 

(2)  If  OPEN  Is  empty,  exit  with  failure;  no  solution  exists. 

(3)  Select  from  OPEN  a node  i at  which  f"  is  minimum.  If  several  nodes  qualify, 
choose  a goal  node  if  there  is  one,  and  otherwise  choose  among  them 
arbitrarily. 

(4)  Remove  node  I from  OPEN  and  place  It  on  a list,  called  CLOSED,  of  expanded 
nodes. 

(5)  If  I Is  a goal  node,  exit  with  success;  a solution  has  been  found. 

(6)  Expand  node  I,  creating  nodes  for  all  Its  successors.  For  every  successor 
node  j of  I: 

(a)  Calculate  f*(J). 

(b)  If  J Is  neither  In  list  OPEN  nor  in  CLOSED,  then  add  It  to  OPEN,  with  Its 
f*  value.  Attach  a pointer  from  j back  to  Its  predecessor  I (In  order 
to  trace  back  a solution  path  onco  a goal  nodo  is  found). 

(c)  If  J was  alroady  on  either  OPEN  or  CLOSED,  compare  tho  f*  value  just 
calculated  for  J with  tho  value  previously  associated  with  the  node. 

If  the  new  value  is  lower,  then  (i)  substitute  it  for  the  old  value, 

(ii)  point  j back  to  I Instead  of  to  its  previously  found  predecessor, 
and  (III)  If  node  J was  on  the  CLOSED  list,  move  It  back  to  OPEN. 

(7)  Go  to  2. 

Step  6c  Is  necessary  for  general  graphs,  in  which  a nodo  can  have  more  than  one 
predecessor.  The  predecessor  yielding  the  smaller  value  of  f*(j)  Is  chosen.  For  trees,  In 
which  a node  has  at  most  one  predecessor,  step  6c  can  bo  Ignored.  Note  that  oven  If  tho 
search  apace  Is  a general  graph,  the  subgraph  that  Is  made  explicit  Is  always  a tree  since 
node  j never  records  more  than  one  predecessor  at  a time. 

Breadth-first,  uniform-cost,  and  depth-first  search  (Article  Cl,  Blind  State-space 
Search)  are  all  special  casos  of  the  ordered  search  technique.  For  breadth-first  search,  we 
choose  f "(I)  to  be  the  depth  of  node  I.  For  uniform-cost  search,  f"(l)  Is  the  cost  of  the  path 
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from  the  start  node  to  node  1.  A depth-first  soarch  (without  a depth  bound)  can  be  obtained 
by  taking  f*(i)  to  be  the  nogative  of  the  depth  of  the  node. 

The  purpose  of  ordered  search,  of  courso,  is  to  reduce  tho  number  of  nodes  expanded 
as  compared  to  blind-search  algorithms.  Its  effectiveness  In  doing  this  depends  directly  on 
the  choice  of  f,  which  should  discriminate  sharply  bctwoen  promising  and  unpromising  nodes. 
If  the  discrimination  is  inaccurato,  howovor,  tho  ordered  search  may  miss  an  optimal  solution 
or  all  solutions.  If  no  exact  moasure  of  promise  is  available,  therefore,  tho  choice  of  f* 
involves  a trade-off  between  time  and  space  on  tho  ono  hand  and  the  guarantee  of  an 
optimal  solution,  or  any  solution,  on  the  other. 


Problem  Types  and  the  Choice  of  f* 

The  moasure  of  a node's  promise--and  consequently,  tiro  appropriateness  of  a 
particular  evaluation  function--deponds  on  tho  problem  at  hand.  Several  cases  can  bo 
distinguished  by  the  type  of  solution  thoy  require.  In  ono.  It  is  assumed  that  the  state  space 
contains  multlplo  solution  paths  with  different  costs;  tho  problem  is  to  find  tho  optimal  (i.e., 
minimum  cost)  solution.  This  first  case  is  well  understood;  see  Article  C3b  on  the  A* 
algorithm. 

The  second  situation  Is  similar  to  the  first  but  with  an  added  condition:  The  problem  is 
hard  enough  that,  If  It  is  treated  as  an  instance  of  case  one,  tho  search  will  probably  exceod 
bounds  of  time  and  space  before  finding  a solution.  The  key  questions  for  case  two  are 
(a)  how  to  find  good  (but  not  optimal)  solutions  with  reasonable  amounts  of  search  effort 
and  (b)  how  to  bound  both  the  soarch  effort  and  the  extent  to  which  the  solution  produced 
is  less  than  optimal. 

A third  kind  of  problem  Is  ono  in  which  there  is  no  concern  for  the  optimality  of  the 
solution;  perhaps  only  one  solution  exists,  or  any  solution  is  as  good  as  any  other.  The 
question  here  is  how  to  minimize  tho  soarch  effort--instead  of,  as  in  case  two,  trying  to 
minimize  some  combination  of  search  effort  and  solution  cost. 

An  example  of  cose  three  comes  from  thoorem  proving,  where  one  may  well  be 
satisfied  with  the  most  easily  found  proof,  however  inelegant.  A clear  example  of  case  two 
Is  the  traveling-salesman  problem,  in  which  finding  some  circuit  through  a set  of  cities  Is 
trivial,  and  the  difficulty,  which  Is  very  great,  Is  entirely  in  finding  a shortest  or  close-to- 
shortest  path.  Most  treatments,  however,  do  not  clearly  distinguish  between  the  two  cases. 
A popular  test  problem,  the  8-puzzle,  can  be  treated  as  being  In  either  class.  For  further 
discussion  of  cases  two  and  three,  see  Article  C3c,  Rolaxlng  tho  Optimality  Requirement. 
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C3b.  A*— Optimal  Search  for  an  Optimal  Solution 

The  A*  algorithm,  described  by  Hart,  Nilsson,  and  Raphael  (1968),  addresses  the 
problem  of  finding  a minimal  cost  path  Joining  the  start  node  and  a goal  node  in  a state-space 
graph.  This  problem  subsumes  the  problem  of  finding  the  path  between  such  nodes 
containing  the  smallest  number  of  arcs.  In  the  latter  problem,  each  arc  (representing  the 
application  of  an  operator)  has  cost  1;  in  the  minimal  cost  path  problem,  the  costs  associated 
with  arcs  can  be  aibitrary.  Historically,  the  predecessors  of  A"  Include  Oijkstra's  algorithm 
(1969)  and  Moore  s algorithm  (1959).  A class  of  algorithms  similar  to  A*  Is  used  in 
operations  research  unuer  the  r.ame  branch-and-bound  algorithms  (see  Hall,  1971;  Hillier  & 

Lleberman,  1974;  Lawler  & Wood,  1966;  and  Reingold,  Nievergelt,  & Deo,  1977). 

The  algorithm  used  by  A*  is  an  ordered  state-space  search  (Article  C3a).  Its  distinctive 
feature  Is  Its  definition  of  the  evaluation  function  f".  As  In  the  usual  ordered  search,  the  node  ' 

chosen  tor  expansion  Is  always  one  at  which  f*  is  minimum. 

t 

Since  f*  evaluates  nodes  in  light  of  the  need  to  find  a minimal  cost  solution,  it  considers 
the  value  of  each  node  n as  having  two  components:  the  cost  of  reaching  n from  the  start 
node,  and  the  cost  of  reaching  a goal  from  node  n.  Accordingly,  f*  Is  defined  by 

f"(n)  * gm(n)  + h*(n)  r 

where  g*  estimates  the  minimum  cost  of  a path  from  the  start  node  to  node  n.  and  h* 

estimates  the  minimum  cost  from  node  n to  a goal.  The  value  f*(n)  thus  estimates  the  minimal  • 

cost  of  a solution  path  passing  through  node  n.  The  actual  costs,  which  P,  g\  and  h*  only 

estimate,  are  denoted  by  f,  g,  and  h,  respectively.  It  is  assumed  that  all  arc  costs  are 

positive. 

The  function  g*.  applied  to  a node  n being  considered  for  expansion.  Is  calculated  as 
the  actual  cost  from  the  start  node  s to  n along  the  cheapest  path  found  so  far  by  the 
algorithm.  If  the  state  space  Is  a tree,  then  g*  gives  a perfect  estimate  since  only  one  path 
from  s to  n exists.  In  a general  state-space  graph,  g*  can  err  only  In  the  direction  of 
overestimating  the  minimal  cost;  its  value  is  adjusted  downward  if  a shorter  path  to  n is 
found.  Even  In  a general  graph,  there  are  certain  conditions  (mentioned  below)  under  which 
g*(n)  can  be  shown  to  be  a perfect  estimate  by  the  time  node  n is  chosen  for  expansion. 

The  function  h"  Is  tho  carrier  of  heuristic  information  and  can  be  defined  in  any  way 
appropriate  to  the  problem  domain.  For  the  interesting  properties  of  the  A’  algorithm  to  hold, 
however,  h*  should  be  nonnegative,  and  it  should  never  overestimate  the  cost  of  reaching  a 
goal  node  from  the  node  being  evaluated.  That  Is,  for  any  such  node  n it  should  always  hold 
that  h*(n)  is  less  than  or  equal  to  h(n),  tho  actual  cost  of  an  optimal  path  from  n to  a goal 
node.  This  last  condition  Is  called  the  admissibility  condition. 


Admissibility  and  Optimality  of  A* 

It  can  be  shown  that  if  h*  satisfies  the  admissibility  condition  and  if,  in  addition,  all  arc 
costs  are  positive  and  can  be  bounded  from  below  by  a positive  number,  then  A*  Is 
guaranteed  to  find  a solution  path  of  minimal  cost  if  any  solution  path  exists.  This  property  Is 
called  the  property  of  admissibility. 
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Although  the  admissibility  condition  requires  h*  to  be  a lower  bound  on  h,  It  is  to  be 
expected  that  the  more  nearly  h*  approximates  h,  tho  bettor  the  algorithm  will  perform.  If  h* 
were  Identically  equal  to  h,  an  optimal  solution  path  would  be  found  without  ever  expanding  a 
node  off  tho  path  (assuming  only  ono  optimal  solution  exists).  If  h*  is  Identically  zero,  A* 
reduces  to  the  blind  uniform-cost  algorithm  (Articlo  Cl).  Two  otherwise  similar  algorithms,  say 
Al  and  A 2,  can  be  compared  with  respect  to  their  choices  of  the  h*  function,  say  hi*  and 
h2*.  Algorithm  At  Is  said  to  be  more  informed  than  A 2 If,  whenever  a node  n (other  than  a goal 
node)  Is  evaluated, 


hi  *(n)  > h2*(n)  . 

On  this  basis  an  optimality  result  for  A*  can  be  stated:  If  A and  A*  are  admissible  algorithms 
such  that  A*  Is  more  Informed  than  A,  then  A*  never  expands  a node  that  Is  not  also 
expanded  by  A.  A proof  (correcting  the  proof  given  in  Nilsson,  1071)  appears  In  Gelperin 
(1977). 


Optimality  and  Heuristic  Power 

The  sense  In  which  A*  yields  an  optimal  search  has  to  do  only  with  the  number  of  nodes 
It  expands  In  the  course  of  finding  a minimal-cost  solution.  But  there  are  other  relevant 
considerations.  First,  the  difficulty  of  computing  h*  also  affects  the  total  computational 
effort.  Second,  It  may  bo  less  Important  to  find  a solution  whose  cost  Is  absolutely  minimum 
than  to  find  a solution  of  reasonable  cost  within  a search  of  moderate  length.  In  such  a case 
one  might  prefer  an  h*  that  evaluates  nodes  more  accurately  in  most  cases  but  sometimes 
overestimates  the  distance  to  a goal,  thus  yielding  an  inadmissible  algorithm.  (See  Article 
C3c.)  The  choice  of  h*  and  the  resulting  heuristic  power  of  the  algorithm  depend  upon  a 
compromise  among  these  considerations. 

A final  question  ono  might  considor  is  the  number  of  node  expansions,  as  opposed  to 
the  numbor  of  distinct  nodes  expanded  by  A*.  Tho  two  totals  will  be  the  same  provided  that 
whenever  a node  n Is  expanded  (moved  to  the  CLOSED  list),  an  optimal  path  to  n has  already 
been  found.  This  condition  Is  always  satisfied  In  a state-space  tree,  where  g*(n)  ■ g(n) 
necessarily.  It  will  also  be  satisfied  in  a general  state-space  graph  If  a condition  called  the 
consistency  assumption  holds  (seo  Hart,  Nilsson,  & Raphael,  1068).  The  general  idea  of  the 
assumption  Is  that  a form  of  the  triangle  Inequality  holds  throughout  the  search  space. 
Specifically,  the  assumption  Is  that  for  any  nodes  m and  n,  the  estimated  distance  h*(m)  from 
m to  a goal  should  always  be  less  than  or  equal  to  the  actual  distance  from  m to  n plus  the 
estimated  remaining  distance,  h*(n),  from  n to  a goal.  For  an  h*  not  satisfying  the 
consistency  assumption  on  a general  state-space  graph,  Martelli  (1977)  has  shown  that  A* 
Is  not  optimal  with  respect  to  tho  number  of  expansions  and  has  given  an  algorithm  that  runs 
more  efficiently  under  these  circumstances. 
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C3c.  Relaxing  the  Optimality  Requirement 

The  A*  algorithm  (C3b)  is  an  ordered  state-space  search  using  the  evaluation  function 
f*  ■ g*  ♦ h\  If  tho  appropriate  conditions  aro  met,  Including  most  Importantly  the  admissibility 
condition,  that  the  estimate  ha(n)  Is  always  less  than  or  equal  to  h(n),  then  A*  Is  guaranteed 
to  find  an  optimal  solution  path  If  one  exists.  Again  under  suitable  conditions,  tho 
performance  of  A*  is  optimal  in  comparison  with  other  similarly  defined  admissible  algorithms. 
Still,  several  questions  remain: 

(1)  One  may  be  moro  concerned  with  minimizing  soarch  effort  than  with  minimizing 
solution  cost.  Is  f*  ■ g*  ♦ h*  an  appropriate  evaluation  function  In  this  case? 

(2)  Even  If  solution  cost  Is  Important,  tho  combinatorics  of  the  problem  may  be 
such  that  an  admissible  A*  cannot  run  to  termination.  Can  speed  be  gained 
at  the  cost  of  a bounded  docrease  In  solution  quality? 

(3)  It  may  be  hard  to  find  a good  heuristic  function  h*  that  satisfies  the 
admissibility  condition;  with  a poor  but  admissible  heuristic  function,  A* 
deteriorates  Into  blind  search.  How  is  the  search  affected  by  an  Inadmissible 
heuristic  function? 


Minimizing  Search  Effort 

An  approach  to  the  first  question  can  bo  stated  as  follows.  The  reason  for  Including  g* 
In  the  evaluation  function  Is  to  add  a breadth-first  component  to  tho  soarch;  without  g\  tho 
evaluation  function  would  estimate,  at  any  node  n,  the  remaining  distance  to  a goal  and  would 
Ignore  the  distance  already  covered  in  reaching  n.  If  tho  object  is  to  minimize  search  of  fort 
instead  of  solution  cost,  one  might  conclude  that  g’  should  be  omitted  from  the  evaluation 
function.  An  early  heuristic  search  algorithm  that  did  just  this  was  Doran  and  Mlchie's  Graph 
Traverser  (Doran  & Michie,  1066;  Doran.  1 06 7 );  tho  evaluation  function  used  was  of  tho 
form  f*  » h",  and  the  object  was  to  minimize  total  soarch  effort  In  finding  solutions  to  tho  8- 
puzzle  and  other  probloms.  A generalization  covering  the  Graph  Traversor  algorithm.  A*,  and 
others  has  been  defined  by  Pohl  (1969,  1970a,  1970b)  as  the  Heuristic  Path  Algorithm 
(HPA).  This  algorithm  gives  an  ordered  state-space  search  with  an  evaluation  function  of  the 
form 


f*  * (1  - w)g*  ♦ wh* 

where  w Is  a constant  In  [0,  1]  giving  tho  relative  Importance  to  be  attached  to  g and  h. 
Choosing  w » 1 gives  tho  Graph  Traversor  algorithm;  w * 0 gives  breadth-first  search;  and 
w « .6  Is  equivalent  to  tho  A*  function  f*  ■ g*  ♦ h\ 

Pohl's  results  concerning  HPA  Indicate  that,  at  least  In  special  cases,  omitting  g*  from 
the  evaluation  function  is  a mlstako.  One  caso  is  that  In  which  h*  Is  tho  most  accurate 
heuristic  function  possible:  If  h*(n)  ■ h(n)  at  overy  node  n,  the  evaluation  function  f*  = h* 
still  expands  no  fewer  nodos  than  f*  * g*  ♦ h\  Tho  other  case  assumes  a simplified  state 
space,  whose  graph  is  an  infinite  m-ary  tree,  and  assumes  that  the  error  In  h*— which  may 
underestimate  or  overestimate  h--ls  bounded  by  a nonnegativo  Integer  e.  In  this  situation  It 
is  shown  that  the  maximum  number  of  nodos  expanded  with  f*  ■ h*  is  greater  than  tho 
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maximum  number  expanded  with  f*  = g*  ♦ h\  and  that  the  difference  between  the  maxima  Is 
exponential  In  the  error  bound  e.  This  analysis  by  Pohl  is  one  of  the  earliest  applications  of 
oracle  or  adversary  analysis  for  discovering  worst-case  algorithmic  efficiency.  As  such  it  is 
an  Important  precursor  to  work  on  NP-complete  problems  and  their  attempted  solution  by 
heuristics.  (For  a genera)  introduction  to  NP-complotenoss  see  Aho,  Hopcroft,  A Ullman, 
1 074.) 


The  two  functions  f*  * h*  and  f*  s g*  ♦ h«  havo  not  been  analyzed  with  respect  to  their 
average-case,  as  opposed  to  worst-caso,  behavior.  Pohl's  empirical  results  suggest  that 
ordered  search  may  typically  expand  the  fewest  nodes,  provided  the  h*  function  is  fairly 
good,  If  g*  is  included  but  given  less  woight  than  h*--that  is,  with  w greater  than  .6  but  less 
than  1.  These  results  were  obtained  for  the  15-puzzle,  a task  exactly  like  the  8-puzzle 
except  that  it  uses  16  tiles  in  a 4 x 4 array. 

For  problems  that  differ  from  the  16-puzzle,  in  that  some  states  load  to  dead  ends 
rather  than  only  to  longer  solutions,  a somewhat  different  approach  has  been  taken  recently 
by  Simon  and  Kadane  (1976).  Whereas  the  evaluation  functions  f*  8 g*  ♦ h*  and  f*  = h*  are 
based  on  the  estimated  solution  cost  at  a given  node.  Simon  and  Kadane  propose  that  the 
function  should  also  take  explicit  account  of  tho  probability  that  the  node  Is  in  fact  on  a 
solution  path.  With  such  a function,  an  expected  long  search  with  high  probability  of 
success  could  readily  rate  just  as  favorably  as  one  that  is  potentially  shorter  but  which  has 
a higher  chance  of  falling. 


Solution  Quality  and  Heuristic  Error 

The  second  question,  of  speed  vs.  solution  quality,  has  been  studied  by  Pohl  (1973, 
1977)  and  Harris  (1973,  1074).  Harris's  work  concerns  the  third  question  (inadmissible 
heuristic  functions)  as  well,  as  do  Pohl's  results  summarized  above.  Both  Harris  and  Pohl 
consider  the  traveling- salesman  problem,  which  is  NP-complete  (Karp,  1972). 

Pohl's  approach  is  a furthor  generalization  of  tho  HPA  evaluation  function:  Now 
f*(n)  * g*(n)  ♦ w(n)h*(n).  That  Is,  the  relative  weight  w to  be  attached  to  g*  and  h*  is  no 
longer  constant;  the  function  w(n),  which  may  bo  greater  than  or  equal  to  1,  is  defined  to 
vary  with  the  depth  of  node  n.  This  approach  is  callod  dynamic  weighting.  With  a definition  of 
w that  weights  h*  less  heavily  as  tho  search  goes  deeper,  and  with  the  assumption  that  h*  Is 
a lower  bound  on  h,  Pohl  shows  that  HPA  will  find  a solution  to  the  traveling-salesman  problem 
whose  cost  Is  bounded  by  the  ratio 


cost  of  tour  found 

— — < 1 ♦ e 

cost  of  optimal  solution 

where  e Is  a constant  In  [0,1)  which  appears  in  the  definition  of  w. 

Dynamic  weighting  was  tested  on  an  instance  of  the  traveling-salesman  problem,  known 
as  the  Croes  problem,  which  involves  20  cities  and  has  a known  optimal  solution  cost  of  246. 
An  admissible  A*— which  produces  an  optimal  solution  If  It  produces  any-had  still  not 
terminated  after  expanding  600  nodes.  With  dynamic  weighting,  however,  together  with  an 
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appropriate  choice  of  e and  the  same  h*  function,  a solution  with  cost  260  was  found  by 
expanding  only  63  nodes. 

Harris's  approach,  called  bandwidth  search,  is  somewhat  different  from  Pohl's.  It 
assumes  that  no  good  h*  function  satisfying  the  admissibility  condition  is  available.  In  its 
place,  he  introduces  the  bandwidth' condition,  which  requires  that  for  all  non-goal  nodes  n, 

(1 ) IV'tn)  s h(n)  ♦ e 
and 

(2)  h(n)  - d s h*(n)  . 

It  Is  assumed  that  h*  satisfies  the  consistency  assumption  (see  Article  C3b). 

With  respect  to  the  first  part  of  the  condition,  it  can  be  shown  that  If  h*  never 
overestimates  the  distance  to  a goal  by  more  than  e,  the  cost  of  a solution  found  by  A*  will 
not  exceed  the  cost  of  an  optimal  solution  by  more  than  e.  With  such  an  h\  the  algorithm  is 
said  to  be  e-admlssible ; and  the  goal  It  finds,  e-optimal. 

Once  the  bandwidth  search  finds  some  solution,  a further  application  of  condition  (1) 
may  show  that  the  cost  of  the  solution  found  is  in  fact  closer  than  e to  an  optimal  solution. 
This  is  possible  because  (a)  the  cost  of  the  solution  found  is  known,  and  (b)  a lower  bound  on 
the  cost  of  every  other  solution  Is  the  minimum,  over  all  nodes  n remaining  on  the  OPEN  list,  of 
f"(n)  - e.  If  the  difference  between  these  two  quantities  is  too  big,  the  search  can  be 
continued  until  it  finds  a solution  that  is  acceptably  close  to  the  optimum. 

The  second  part  of  the  bandwidth  condition,  condition  (2),  can  be  used  to  save  storage 
space  by  dropping  nodes  from  the  OPEN  list,  without  any  risk  of  dropping  a node  that  is  in 
fact  on  an  optimal  path  to  a goal.  Let  node  q be  a node  that,  having  a minimum  value  of  f\ 
has  been  selected  for  expansion.  Then  any  node  m may  safely  be  dropped  from  OPEN  if 
f*(m)  is  hopelessly  big  compared  to  f(q).  Specifically,  it  can  be  shown  that  all  nodes  m can 
be  dropped  if  there  Is  a node  q such  that 

f*(m)  - (e  + d)  > f*(q)  . 

Harris  notes  that  it  may  be  difficult  to  find  a heuristic  function  h*  that  satisfies  both 
parts  of  the  bandwidth  condition.  One  may  instead  define  two  heuristic  functions,  one  to 
order  the  search  and  one  to  determine  which  nodes  can  be  dropped.  Such  functions,  say  hi  * 
and  h2",  should  then  satisfy 


and 


(1‘)  h1*(n)  s h(n)  ♦ e 
(2*)  h(n)  - d s h2*(n)  . 


Using  two  such  heuristic  functions,  Harris  tested  the  bandwidth  search  on  several 
instances  of  the  traveling-salesman  problem  including  the  20-city  Croes  problem  mentioned 
above.  Harris's  results.  Including  a comparison  with  A*  using  an  admissible  heuristic  function, 
are  summarized  below.  The  OPEN  list  was  limited  to  500  nodes. 
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BANDWIDTH 

SEARCH 

A0MISSI8LE  SEARCH 

No.  of 
cities 

Quality 
of  solution 

Nodes 

expanded 

Quality 
of  solution 

Nodes 

expanded 

6 

5-optlmal 

6 

6 

optimal 

14 

optimal 

18 

11 

optimal 

14 

none 

500  open  nodes 

20 

4-optlmal 

42 

none 

500  open  nodes 

Figure  1.  Comparison  of  bandwidth  search  and  admissible 

search. 
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C3d.  Bidlrtctional  Starch 

Earlier  articles  in  this  chapter  describe  (a)  heuristic  state-space  search  methods 
using  forward  reasoning  and  (b)  a blind  state-space  search  combining  forward  and  backward 
reasoning  Into  a bidirectional  algorithm.  The  kinds  of  problems  to  which  a bidirectional  state- 
space  method  applies  are  considered  in  Article  Cl;  In  general,  it  must  be  possible  In  these 
problems  to  search  either  forward,  from  the  Initial  state  toward  the  goal,  or  backward,  from 
the  goal  toward  the  initial  state.  A bidirectional  search  pursues  both  lines  of  reasoning  in 
parallel,  growing  two  search  trees  and  terminating  when  they  meet.  The  motivation  is  that,  In 
many  cases,  the  number  of  nodes  in  a search  tree  grows  exponentially  with  its  depth;  If  a 
solution  can  be  found  by  using  two  trees  of  half  the  depth,  the  search  effort  should  be 
reduced  significantly.  Blind  bidirectional  search  was  in  fact  found  to  expand  far  fewer  nodes 
than  its  unidirectional  counterpart.  A natural  next  question  is  whether  heuristic  bidirectional 
search  can  give  still  greater  Improvements  in  efficiency. 

This  question  was  investigated  by  Pohl  (1969,  1971).  Whereas  his  blind  bidirectional 
algorithm  used  forward  and  backward  uniform-cost  search,  his  heuristic  algorithm  used 
forward  and  backward  ordered  search.  Otherwise,  the  two  algorithms  differed  mainly  in  their 
termination  conditions.  In  both  cases  the  termination  condition  was  complicated  by  the  fact 
that  the  algorithms  were  designed  to  find  an  optimal  path  between  the  start  and  goal  nodes; 
they  could  be  simplified  if  any  path  would  do. 

As  evolution  functions,  Pohl's  heuristic  bidirectional  algorithm  used  functions  parallel  to 
those  of  A*.  For  a node  x In  the  forward  search  tree: 

gs(x)  measured  the  shortest  path  found  so  far  from  the  start  node,  s, 
to  x; 

hs(x)  estimated  the  minimum  remaining  distance  from  x to  the  terminal 
node,  t;  and 

fs(x)  ■ gs(x)  ♦ hs(x)  was  the  evaluation  function. 

Similarly,  for  a node  x generated  in  the  backward  search: 

gt(x)  measured  the  shortest  path  found  so  far  from  x to  t; 

ht(x)  estimated  the  minimum  distance  from  s to  x;  and 

ft(x)  ■ gt(x)  ♦ ht(x)  was  the  evaluation  function. 

Constraints  were  placed  on  the  heuristic  functions  hs  and  ht,  corresponding  to  the 
admissibility  condition  and  the  consistency  assumption  of  A",  in  order  to  guarantee  the 
optimality  of  the  solution. 

Pohl's  results,  In  experiments  using  bidirectional  heuristic  search  on  the  15-puzzle, 
were  disappointing.  It  was  hoped  that  the  search  trees  rooted  at  the  start  and  goal  nodes 
would  meet  near  the  middle  of  the  solution  path.  In  blind  search,  this  had  happened 
necessarily  because  both  trees  were  expanded  breadth-first.  (Recall  that  uniform-cn^: 
search  Is  a generalization  of  the  breadth-first  algorithm.)  In  the  heuristic  case,  however,  u. 
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search  In  each  direction  was  narrowed.  Since  each  problem  had  many  alternate  solutions, 
the  typical  outcome  was  that  both  search  trees  grew  to  Include  nearly  complete  but 
different,  solution  paths  before  Intersecting. 

Several  ideas  have  been  advanced  for  forcing  the  trees  to  meet  earlier  while  retalninq 
the  benefit  of  heuristic  information  (Pohl.  1971;  Kowalski,  1972;  de  Champeaux  & Sint. 
1976,  1977;  Pohl,  1977).  One  that  has  boon  tested  Is  that  of  Champeaux  and  Sint,  which 
redefines  the  heuristic  functions  hs  and  ht  as  follows: 

Let  T-OPEN  be  the  list  of  unexpanded  nodes  of  the  backward  search 
tree.  For  a node  x in  the  forward  search  tree,  hs(x)  estimates  the 
minimum  distance  from  x to  the  goal  t by  way  of  some  node  y In  T- 
OPEN.  That  Is,  hs(x)  is  the  minimum,  over  all  nodes  y on  T-OPEN,  of 
the  estimated  distance  from  x to  y plus  gt(y),  the  length  of  the 
shortest  known  path  from  y to  the  goal. 

The  function  ht  is  defined  analogously.  The  authors  reported,  for  the  same  problems  Pohl  had 
used,  that  the  algorithm  generally  produced  shorter  solution  paths,  with  fewer  nodes 
expanded,  and  that  the  search  graphs  now  did  meet  near  the  middle  of  the  search  spare. 
Unfortunately,  however,  hs  and  ht  were  so  expensive  to  compute--since  for  each  node  x to 

be  expanded,  Its  distance  must  be  estimated  to  every  node  y on  the  opposite  OPEN  list 

that  the  algorithm  still  ran  much  more  slowly  than  unidirectional  heuristic  search. 


References 

See  de  Champeaux  & Sint  (1976),  de  Champeaux  & Sint  (1977),  Kowalski  (1972)  Pohl 
(1060),  Pohl  (1071),  and  Pohl  (1077). 
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C4.  Heuristic  Search  of  an  AND/OR  Graph 

This  article  returns  to  the  problem  of  searching  an  AND/OR  graph,  as  opposed  to  an 
ordinary  state-space  graph.  The  distinction  between  the  two  is  the  presence  of  AND  nodes, 
which  add  conceptual  complications  to  the  search  problem.  Each  node  of  the  AND/OR  graph 
represents  a goal  to  be  achieved.  It  will  be  assumed  throughout  that  reasoning  Is  backward, 
from  an  Initial  goal  (the  root)  toward  an  equivalent  set  of  subgoals,  ell  of  which  have 
immediate  solutions.  On  this  assumption,  an  AND/OR  graph  constitutes  (In  the  terminology  of 
this  chapter)  a problem-reduction  representation.  This  Identification  gives  another  way  of 
stating  the  distinction  between  problem-reduction  and  state-space  representations:  State- 
space  operators  always  take  exactly  one  input  and  produce  exactly  one  output;  a problem- 
reduction  operator  also  takes  a single  input  but  may  produce  multiple  outputs  (see  Section 
B). 

To  put  the  matter  further  into  perspective,  one  may  also  conceive  of  searching  an 
AND/OR  graph  In  the  forward  direction--from  the  primitive  problems,  whose  solutions  are 
already  known,  toward  the  problem  one  actually  wishes  to  solve.  Just  such  a graph  search  is 
that  typically  conducted  by  a resolution  theorem-prover,  as  it  brings  together  two  or  more 
axioms  or  previous  conclusions  and  applies  to  them  an  operator  yielding  one  new  deduction 
as  Its  result.  (See  Theorem  Provinfl.)  Forward  reasoning  In  an  AND/OR  graph,  then,  would  be 
distinguished  from  a state-space  search  by  the  presence  of  multiple-input,  single-output 
operators.  For  further  discussion,  including  an  algorithm  for  bidirectional  search  of  an  AND/OR 
graph,  see  Kowalski  (1972);  see  also  Martelli  and  Montanarl  (1973). 

The  search  of  an  AND/OR  graph  using  backward  reasoning  raises  numerous  problems. 
Previous  articles  (B2  and  C2)  have  considered 

(a)  what  constitutes  a solution  subgraph  of  an  AND/OR  graph,  and 

(b)  blind  search  algorithms  for  finding  a solution  subgraph. 

This  article  considers  three  additional  problems: 

(c)  What  might  one  mean  by  an  optimal  solution  subgraph? 

(d)  How  can  heuristic  information  be  brought  to  bear  on  the  search  for  an 
optimal  solution? 

(e)  What  limitations  are  there  on  AND/OR  graphs  and  the  associated 
search  algorithms  as  general  tools  for  problem  solving? 


The  Definition  of  an  Optimal  Solution 

A solution  of  an  AND/OR  graph  Is  a subgraph  demonstrating  that  the  start  node  is 
solved.  As  In  a state-space  search,  one  may  ask  for  a solution  of  minimal  cost.  The  cost  of 
a solution  tree  can  be  defined  in  either  of  two  ways  (Nilsson,  1971): 
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The  sum  cost  of  a solution  tree  is  the  sum  of  all  arc  costs  In  the  tree. 

The  max  cost  of  a solution  tree  is  the  sum  of  arc  costs  along  the  most 
expensive  path  from  the  root  to  a terminal  node. 

For  example,  if  every  arc  in  the  solution  tree  has  cost  1.  then  the  sum  cost  is  the  number  of 
arcs  in  the  tree,  and  the  max  cost  is  the  depth  of  the  deepest  node. 

If  the  entire  search  space  had  been  explored,  then  an  optimal  solution  tree  could  be 
constructed  and  its  cost  measured  as  follows.  Let  c(n,m)  be  the  cost  of  the  arc  from  node  n 
to  a successor  node  m.  Define  a function  h(n)  by; 

If  n Is  a terminal  node  (a  primitive  problem),  then  h(n)  = 0. 

If  n has  OR  successors,  then  h(n)  is  the  minimum,  over  all  its 
successors  m,  of  c(n,m)  + h(m). 

If  n has  AND  successors  and  sum  costs  are  used,  then  h(n)  is  the 
summation,  over  all  successors  m,  of  c(n,m)  ♦ h(m). 

If  n has  AND  successors  and  max  costs  are  used,  then  h(n)  is  the 
maximum,  over  all  successors  m,  of  c(n,m)  h(m). 

If  n Is  a nonterminal  node  with  no  successors,  then  h(n)  Is  infinite. 

According  to  this  definition,  h(n)  is  finite  if  and  only  if  the  problem  represented  by  node  n is 
solvable.  For  each  solvable  node  n,  h(n)  gives  the  cost  of  an  optimal  solution  tree  for  the 
problem  represented  by  node  n.  If  s is  the  start  node,  then  h(s)  Is  the  cost  of  an  optimal 
solution  to  the  Initial  problem. 

Consider,  for  example,  the  AND/OR  tree  of  Figure  1,  with  arc  costs  as  indicated.  Each 
node  without  auccessors  is  marked  t or  u according  to  whether  it  Is  terminal  or  unsolvable. 


S 


Figure  1 . An  AND/OR  tree. 
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If  sum  costs  are  used,  the  values  of  h are  as  shown  In  Figure  2.  and  the  optimal  solution  is  the 
subgraph  comprising  nodes  S,  B,  D,  E,  t6,  and  t6.  The  abbreviation  inf  denotes  infinity. 
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Figure  2.  Sum  costs. 


If  max  costs  are  used,  then  the  values  of  h are  as  shown  in  Figure  3,  and  the  optimal  solution 
is  the  subgraph  comprising  nodes  S,  A,  tl,  t2,  and  t3. 
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Figure  3.  Max  costs. 
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Ordered  Search  Algorithms  for  on  AND/OR  Graph 

In  an  ordered  state-space  search,  ono  may  use  an  (valuation  Junction  t*  that,  applied  to 
nodo  n,  roturns  tbo  estimated  minimum  cost  of  a solution  path  passing  through  node  n.  1 ho 
noxt  nodo  oxpondod  Is  always  ono  at  which  f*  is  minimum— that  Is,  ono  oxtonds  tho  most 
promising  potential  solution  path.  I ho  successors  of  nodo  n nra  now  nodes,  but  ono  could 
just  ns  well  think  of  thorn  ns  now  potential  solution  paths,  each  differing  from  n parent 
(potential  solution  path)  by  tho  Inclusion  of  ono  more  stop. 

In  tho  extension  of  heuristic  search  to  AND/OH  graphs,  there  Is  no  longer  a one-to-one 
correspondence  between  tho  choice  of  a nodo  to  expand  and  the  choice  of  a potential 
solution  to  bo  oxtondod.  Consider,  for  example,  tho  search  graph  of  figure  A. 


S 


Figure  A.  An  AND/OR  graph  containing  two 
potential  solution  trees. 

Slnco  C and  D are  OR  nodes,  an  actual  solution  of  node  S will  contain  only  ono  of  them.  To 
expand  node  A Is  thus  to  extend  two  potential  solution  tuts, 


Conversely,  a decision  to  oxtond  tho  potential  solution  troo  on  tho  loft  can  bo  carried  out  by 
expanding  eithor  node  A or  nodo  C.  One  must  bo  clear,  therefere,  about  what  kind  of  object 
tho  expansion  procoss  Is  to  apply  to.  This  decision  will  affect  the  definition  of  tho 
evaluation  function. 

Nilsson's  algorithm.  An  approach  taken  by  Nilsson  (1060,  1071)  selects  Individual 
nodes  to  expand  by  a two-step  process:  I Irst,  Identify  tho  most  promising  potential  solution 
tree;  then  chooso  a nodo  within  that  treo  for  expansion.  1o  accomplish  the  first  step,  an 
evaluation  function  h*  Is  defined  at  every  node  n of  the  troo  that  has  not  been  shown  to  be 
unsolvabla.  This  fwrrctlon  Is  an  estimate  of  h(n);  that  Is,  It  estimates  the  cost  of  an  optimal 
solution  to  tho  problem  at  node  n.  If  n Is  known  to  bo  a terminal  node,  then  by  definition  h*(n) 
■ h(n)  ■ 0.  Otherwise,  If  n has  not  yet  boon  expanded,  then  the  estimate  must  be  based  on 
whatever  heuristic  Information  Is  available  from  tho  problem  domain.  For  example,  In  the 
search  treo  of  Figuro  A,  h*  would  provide  heuristic  estimates  of  tho  cost  of  solving  nodes  A, 
C,  and  D.  Tho  following  rule  thon  permits  h"  to  be  computed  for  each  nodo  whose  successors 
havo  already  boon  generated  (and  to  be  recomputed  as  the  search  troo  Is  expanded): 

If  n has  OH  successors  m,  then  h*(n)  is  the  minimum,  over  these 

successors,  of  c(n,m)  * h*(m). 
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If  n has  AN 0 successors  m and  sum  costs  are  used,  then  h*(n)  Is  the 
summation,  over  those  successors,  of  c(n,m)  «•  h*(m). 

If  n has  AND  successors  m and  max  costs  are  used,  then  h*(n)  is  the 
maximum,  over  these  successors,  of  c(n,m)  ♦ h"(m). 

Finally,  the  most  promising  potential  solution  tree,  T,  Is  defined  In  terms  of  h*: 

The  start  node  s is  In  T. 

If  the  search  tree  (the  part  of  the  search  space  generated  so  far)  contains  a 
node  n and  AND  successors  of  n,  then  all  these  successors  are  In  T. 

If  the  search  tree  contains  a node  n and  OR  successors  m of  n,  then  one 
successor  m Is  In  T such  that  c(n,m)  ♦ h*(m)  Is  minimal. 

The  estimated  cost  of  T Is  h*(s).  If  all  tho  other  potential  solution  trees  for  the  same  search 
tree  were  constructed,  It  would  be  found  that  T is  one  for  which  h*(s)  Is  minimal. 

An  ordered-search  algorithm  for  an  AND/OR  tree  can  now  be  stated  as  follows: 

(1)  Put  the  start  node,  s,  on  a list,  OPEN,  of  unexpanded  nodes. 

(2)  From  the  search  tree  constructed  so  far  (initially,  just  s),  compute  the  most 
promising  potential  solution  tree  T. 

(3)  Select  a node  n that  Is  on  OPEN  and  In  T.  Remove  node  n from  OPEN  and 
place  It  on  a list  callod  CLOSED. 

(4)  If  n Is  a terminal  node,  then 

(a)  Label  node  n solved. 

(b)  If  the  solution  of  n makos  any  of  Its  ancestors  solved,  label  these 
ancestors  solved. 

(c)  If  the  start  node  is  solved,  exit  with  T as  tho  solution  tree. 

(d)  Remove  from  OPEN  any  nodes  with  a solved  ancestor. 

(6)  Otherwise,  If  node  n has  no  successors  (i.e.,  If  no  operator  can  be  applied), 
then 

(a)  Label  node  n unsolvable. 

(b)  If  the  unsolvability  of  n makes  any  of  Its  ancestors  unsolvable,  label 
all  such  ancestors  unsolvable  as  well. 

(c)  If  the  start  node  Is  labeled  unsolvable,  exit  with  failure. 

(d)  Remove  from  OPEN  any  nodes  with  an  unsolvable  ancestor. 

(6)  Otherwise,  expand  node  n,  generating  all  Its  Immediate  successors  and.  for 
each  successor  m representing  a set  of  more  than  one  subproblem, 
generating  successors  of  m corresponding  to  the  individual  subproblems. 
Attach,  to  each  newly  generated  node,  a pointer  back  to  Its  Immediate 
predecessor,  and  compute  h"  for  each  newly  generated  node.  Place  all  the 
new  nodes  that  do  not  yet  have  descendants  on  OPEN.  Finally,  recompute 
h"(n)  and  h*  at  each  ancestor  of  n. 

(7)  Go  to  (2). 


The  ordered-search  algorithm  con  be  shown  to  be  admissiblt— that  Is,  It  will  find  a 
minimum-cost  solution  tree  If  any  solution  exists— provided  that:  (a)  h*(n)  Is  less  than  or 
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equal  to  h(n)  for  oach  open  node  n,  and  (b)  all  arc  costs  are  greater  than  some  small  positive 
number  d.  The  efficiency  of  the  algorithm,  howover,  depends  both  on  the  accuracy  of  h*  and 
on  the  Implementation  of  stop  3.  in  which,  having  found  the  most  promising  potential  solution 
tree  to  expand,  one  must  decide  to  expand  a specific  node  within  that  tree.  If  the  partial 
tree  T Is  In  fact  part  of  an  optimum  solution,  tho  choice  is  immaterial.  If  It  Is  not,  however, 
then  the  best  node  to  expand  would  bo  the  one  that  will  earliest  reveal  the  error. 

Chang  and  Slagle's  algorithm.  A different  approach  has  been  taken  by  Chang  and 
Slagle  (1971).  Here  the  objects  expanded  are  potential  solution  graphs.  A tip  node  in  such  a 
graph  is  any  node  that  does  not  yet  have  successors.  To  expand  the  potential  solution 
graph,  one  expands  all  Its  nonterminal  tip  nodes  at  once  and  then  forms  all  the  now  potential 
solution  graphs  that  result.  Each  graph  is  represented  on  the  OPEN  list  by  the  conjunction  of 
Its  tip  nodes,  representing  a set  of  subproblcms  to  which  the  start  node  can  be  reduced. 

For  example,  suppose  that  expansion  of  tho  initial  graph,  consisting  of  only  the  start 
node  S,  shows  that  S can  be  reduced  to  problems  A and  B or  to  problem  C.  The  OPEN  list 
then  becomes  (A&B,  C).  Assumo  that  A&B  is  solocted  for  expansion,  that  A can  be  reduced 
to  D or  E,  and  that  B can  bo  reduced  to  F or  G.  Thore  are  four  new  potential  solution  trees, 
and  the  OPEN  list  Is  now  (D&F,  D&G,  E&F,  E&G,  C).  The  search  succeeds  when  It  selects  for 
expansion  a potential  solution  graoh  represented  by  a conjunction  of  nodes  all  of  which  are 

terminal.  £ 

The  Chang  and  Slaglo  approach  assimilates  AND/OR  graph  search  to  tho  problem  of 
state-space  search.  Each  distinct  conjunction  of  probloms  to  bo  solved  corresponds  to  a 
distinct  state  of  a state-space  graph.  The  evaluation  function  used,  f*.  Is  also  parallel  to 
the  function  used  In  A\  It  is  defined  by  f*  = g*  ♦ h*  , where  g*  meosures  the  cheapest  way 
found  so  far  to  reduce  the  start  nodo  to  a given  conjunction  of  subproblems  and  h*  estimates 
the  minimum  remaining  cost  of  a graph  sufficient  to  solve  all  those  subproblems. 

The  treatment  of  AND/OR  graph  search  as  an  instance  of  state-space  search  has 
several  consequences.  One  Is  that  tho  search  of  a general  AND/OR  graph,  as  opposed  to  an 
AND/OR  tree,  now  raises  no  special  problems  Another  is  that  the  algorithm  can  be  shown 
(Chang  & Slagle,  1971),  under  appropriate  conditions,  to  be  not  only  admissible  but  also 
optimal  with  respect  to  the  number  of  potential  solution  graphs  expanded.  It  does  not, 
however,  appear  to  be  optimal  (in  some  reasonable  sense  of  that  term)  In  comparison  with 
algorithms  that  expand  only  one  node  at  a time  (see  Kowalski,  1972). 

Interdependent  Subproblems 

The  discussion  so  far  has  assumed  that  whenever  the  start  node  Is  reduced  to  a 
conjunction  of  subproblems,  all  subproblems  can  be  solved  independently,  so  that  the  solution 
to  one  has  no  effect  on  the  solution  to  any  other.  This  assumption  is  frequently  unjustified, 
and  much  of  the  chapter  on  Planning  explores  ways  of  dealing  with  Interacting  subproblems. 

Two  kinds  of  examples,  given  by  Levi  and  Sirovich  (1976,  1976)  with  explicit  reference  to 
the  AND/OR  graph  formalism,  are:  (a)  probloms  requiring  consistent  binding  of  variables  and 
(b)  problems  involving  the  expenditure  of  scarce  resources. 

An  Illustration  of  the  former  Is  tho  well-known  problem  of  showing  that  there  exists  a 
fallible  Greek,  given  that  the  entire  search  space  Is  as  follows: 


— -----  . . i 
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F Ind  d fallible  Greek 


Find  somethin^  fallible  Find  something  Greek 

Find  something  human  Socrates  Is  Greek 

/ \ 

Turing  Socrates 

is  human  Is  human 

Figure  6.  An  AND/OR  graph  requiring  consistent  binding 
of  the  variable  "something." 


An  algorithm  llko  Nilsson's  falls  hero  for  two  reasons.  First,  It  has  no  mechanism  for 
discovering  that  "Turing  is  human"  and  "Socrates  is  Greek"  fail  to  constitute  a solution. 
Second,  even  If  such  a mechanism  were  introduced,  the  algorithm  has  no  moans  for  undoing 
the  solution  to  a subproblem  once  it  has  been  solved.  If  "Turing  Is  human"  Is  the  first 
problem  found  to  be  primitive,  then  "Find  something  human"  and  "Find  something  fallible"  are 
marked  solved;  "Socrates  Is  human"  Is  removed  from  the  OPEN  list  as  no  longer  in  need  of 
consideration;  and  "Find  something  Greek,"  using  the  previous  value  of  "something,"  then 
becomes  unsolvable. 


An  example  of  the  second  typo  of  problom  is  tho  following:  Show  that  John  can  seduce 
the  actress,  given  that  seducing  the  actross  can  be  reduced  to  getting  a car  and  getting  a 
yacht;  and  that  John  has  $6000,  a car  costs  $6000,  and  a yacht  costs  $6000.  Here  either 
of  the  algorithms  given  above  would  wrongly  conclude  that  John  can  seduce  the  actress.  A 
variant  of  the  scarce  resource  problem  arises  In  robot  planning  tasks  (such  as  those 
performed  by  STRIPS,  Article  05),  where  application  of  an  operator  representing  a robot 
action  solving  one  subproblom  may  make  Inapplicable  tho  operator  needed  to  solve  another 
subproblem. 

To  handle  problems  of  those  kinds,  Levi  and  Sirovlch  define  a gtnnalittA  .4 N DIOR 
graph,  which  differs  most  Importantly  from  an  ordinary  AND/OR  graph  In  that  reduction 
operators  are  permitted  to  take  two  or  more  nodes  as  Input.  For  example,  let  R be  a 
resource  that  can  be  used  only  once.  Then  If,  In  tho  standard  formulation,  the  original 
problom  Is  to  accomplish  PI  and  P2,  tho  problom  is  reformulated  as  PI  8.  P2  S>  R.  Suppose 
the  following  reduction  operators  are  available  (whore  ->  moans  "can  be  reduced  to"  and  1 
denotes  a trivial  problem): 


1)  SOPUP2&R 

2)  PI  9.  R ->  T 

3)  PI  ->  P3 

4)  P2  S.  R ->  P3 

6)  P3  ->  T 

6)  R ->  T 


Then  there  la  only  one  solution,  which  is  achieved  using  operators  1,  3,  4,  and  6. 


60 


Al  Handbook 


In  the  ordered  search  of  a generalized  AND/OR  graph,  the  objects  placed  on  the  OPEN 
list  are  potential  solution  graphs,  not  Individual  nodes.  Expansion  of  a potential  solution 
graph  (PSG)  consists  of  applying  all  possible  operators  to  obtain  a new  set  of  PSGs,  each 
differing  from  Its  parent  by  virtue  of  one  additional  operator  application.  If  the  same 
subproblem  occurs  more  than  once  within  a PSG,  each  occurrence  is  represented  by  a 
separate  node.  If  the  same  PSG  is  generated  more  than  once,  later  occurrences  are  simply 
discarded.  Since  distinct  PSGs  are  retained,  alternate  solutions  to  the  same  subproblem  are 
available. 


As  In  the  usual  ordered  search,  the  object  chosen  for  expansion  next  is  always  one 
where  the  evaluation  function  Is  minimum.  The  evaluation  function  is  h*;  for  each  PSG,  It  is 
computed  similarly  to  the  h"  of  Nilsson's  algorithm.  The  value  of  each  potential  solution  graph 
Is  then  the  evaluation  of  the  start  node.  h"(s),  as  computed  for  that  graph.  Both  admissibility 
and  optimality— the  latter  with  respect  to  the  number  of  PSGs  expanded--can  be  shown. 


References 

See  Chang  & Slagle  (1971),  Kowaluki  (1972),  Levi  & Sirovich  (1976),  Levi  & Sirovich 
(1976),  Martelll  & Montanarl  (1973),  Nilsson  (1969),  and  Nilsson  (1971). 
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C6.  Game  Tree  Search 
C6a.  Mlnlmax  Procedure 


Tha  Mlnlmax  Formalism 

The  mlnlmax  procedure  is  a technique  for  searching  game  trees  (Article  B3).  As  a first 
example,  Figure  1 gives  a simple  game  tree  to  which  the  procedure  may  be  applied.  Each 
node  represents  a position  in  the  game.  Nonterminal  nodes  are  labeled  with  the  name  of  the 
player,  A or  8,  who  is  to  move  from  that  position.  It  is  A's  turn,  and  the  problem  is  to  find  his 
best  move  from  position  1.  Exactly  three  moves  remain  in  the  game.  Terminal  nodes  are 
marked  with  their  value  to  player  A by  the  words  "win,"  "lose,"  or  "draw." 


1 

A 


Figure  1.  A game  tree  from  the  standpoint  of 
player  A,  who  Is  to  move  next. 


According  to  the  mlnlmax  technique,  player  A should  move  to  whichever  one  of  positions 
2 or  3 has  the  greater  value  to  him.  Given  the  values  of  the  terminal  positions,  the  value  of 
a nonterminal  position  Is  computed,  by  backing  up  from  the  terminals,  as  follows: 

The  value  to  player  A of  a node  with  OR  successors  (a  node  from 
which  A chooses  the  next  move)  is  the  maximum  value  of  any  of  its 
successors. 

(1) 

The  value  to  A of  a node  with  AND  successors  (a  node  from  which  B 
chooses  the  next  move)  Is  the  minimum  value  of  any  of  Its  successors. 

In  the  example,  node  2 evaluates  to  a loss  for  A (since  B can  then  force  a loss  by  moving  to 
node  6),  and  node  3 evaluates  to  a draw  (since  the  best  B can  then  do  Is  move  to  node  7 or 
0).  It  will  be  noted  that  the  prediction  of  the  opponent's  behavior  assumes  he  Is  also  using 
mlnlmaxi  In  evaluating  a node  with  AND  successors,  A must  assume  that  B will  make  his  best 
possible  move.  The  technique  ignores  the  possibility  that  B might  overlook  his  chance  for  a 


— 
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sure  win  If  A goes  to  node  2.  Similarly,  it  supplies  no  basis  on  which  B might  choose  to  move 
to  node  9 In  preference  to  node  7. 

Because  of  the  way  in  which  nodes  are  evaluated,  player  A (whose  viewpoint  the  tree 
represents)  is  often  called  MAX,  and  player  8,  MIN.  The  names  PLUS  and  MINUS  are  also 
sometimes  used.  If  the  tree  of  Figure  1 were  to  be  evaluated  from  MIN's  standpoint  Instead 
of  MAX'S,  It  would  appear  as  In  Figure  2.  The  AND  and  OR  nodes  are  reversed,  and  the  value 
of  each  node  to  MIN  Is  the  opposite  of  its  value  to  MAX. 


1 

A 

draw 


A A A A A A 


lose  lose  win  lose  win  win  draw  draw  lose  draw  win  draw 


Figure  2.  The  game  tree  of  Figure  1 from  B's  standpoint. 


The  Negmax  Formalism 

Knuth  and  Moore  (1976)  have  given  a game-tree  representation  that  unifies  Figures  1 
and  2 and  conveniently  permits  a single  procedure  to  return  optimal  moves  for  both  players  A 
and  B.  In  this  representation,  the  value  given  each  node  Is  its  value  to  the  player  whose 
turn  It  would  be  to  move  at  that  node.  If  n Is  a terminal  node,  Its  value  Is  an  integer  denoted 
f(n).  (The  value  of  n to  the  other  player  Is  -f(n).)  The  value  of  every  node  Is  then  returned 
by  a function  F defined  as  follows: 

F(n)  ■ f(n),  If  n has  no  successors; 

F(n)  ■ max  (-F(nl),  ...  ,-F(nk)},  if  n has  successors  nl, ...  ,nk. 

The  best  move  for  either  player  is  then  to  a node  with  maximum  value;  that  is,  the  player 
whose  turn  It  Is  at  node  n should  move  from  node  n to  a node  nl  with  - F(nl)  * F(n).  This 
formulation,  which  is  equivalent  to  mlnlmax,  Is  called  negmax.  The  tree  It  produces  for  the 
game  of  Figures  1 and  2 is  shown  in  Figure  3.  The  numerical  value  of  a win  Is  assumed  to  be 
♦1 ; of  a loss,  -1 ; and  of  a draw,  0. 
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Figure  3.  The  game  tree  of  Figure  1 In  NEGMAX  notation. 


Searching  a Partial  Game  Tree 

In  the  above  descriptions  of  the  minimax  and  negmax  algorithms,  it  was  assumed  that  a 
complete  game  tree  had  already  been  generated.  For  most  games,  however,  the  tree  of 
possibilities  is  far  too  large  to  be  generated  fully  and  searched  backward  from  the  terminal 
nodes  for  an  optimal  move.  An  alternative  Is  to  generate  a reasonable  portion  of  the  tree, 
starting  from  the  current  position;  make  a move  on  the  basis  of  this  partial  knowledge;  let  the 
opponent  reply;  and  then  repeat  the  process  beginning  from  the  new  position.  A "reasonable 
portion  of  the  tree"  might  be  taken  to  mean  all  legal  moves  within  a fixed  limit  of  depth,  time, 
or  storage,  or  It  might  be  refined  In  various  ways.  For  discussion  of  the  refinements,  see 
article  C5c. 

Once  the  partial  tree  exists,  minimaxing  requires  a means  for  estimating  the  value  of  its 
lip  nodes,  that  is,  the  nodes  of  the  partial  tree  without  successors.  A function  assigning  such 
a value  is  called  a static  evaluation  function-,  it  serves  a purpose  comparable  to  that  of  the 
heuristic  function  h*  used  In  Nilsson's  ordered  search  of  an  AND/OR  tree  (Article  C4).  If  the 
partial  game  tree  contains  any  nodes  that  are  terminal  for  the  entire  tree,  the  static 
evaluation  function  conventionally  returns  positive  Infinity  for  a win,  negative  infinity  for  a 
loss,  and  zero  for  a draw.  At  other  tip  nodes,  the  function  has  a finite  value  which,  in  the 
minimax  formulation,  is  positive  for  positions  favorable  to  MAX  and  negative  at  positions 
favorable  to  MIN.  The  minimax  procedure  then  assigns  backed-up  values  to  the  ancestors  of 
the  tip  nodes  In  accordance  with  the  rules  given  in  (1)  above.  It  is  assumed  that  the 
backed-up  evaluations  give  a more  accurate  estimate  of  the  true  value  of  MAX'S  possible 
moves  than  would  be  obtained  by  applying  the  static  evaluation  function  directly  to  those 
moves  and  not  looking  ahead  to  their  consequences. 

References 

See  Knuth  & Moore  (1876),  Nilsson  (1971),  Slagle  (1971),  and  Winston  (1977). 
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C6b.  Alpha-beta  Pruning 

The  mlnlmax  procedure  described  In  Article  C5a  decides  on  a best  move  from  node  n,  in 
a full  or  partial  game  treo,  by  evaluating  every  node  in  the  tree  that  descends  from  node  n. 
Frequently,  this  exhaustive  evaluation  is  a waste  of  time.  Two  examples  are  shown  in 
Figures  1 and  2,  Each  node  is  marked  with  the  name  of  the  player  who  is  to  move  from  that 
position. 


1 

MAX 
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Figure  1.  An  alpha  cutoff. 
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Figure  2.  A beta  cutoff. 


In  Figure  1,  nodes  2 and  4 have  been  evaluated  either  by  the  static  evaluation  function 
or  by  backing  up  from  descendants  omitted  from  the  figure.  If  MAX  moves  to  node  2,  he 
achieves  a position  whose  estimated  value  is  15.  If  he  moves  to  node  3,  MIN  can  hold  him  to 
10.  Therefore,  the  value  of  node  3 is  at  most  10,  so  MAX  should  decide  to  move  to  node  2. 
The  important  point  is  that  this  decision  can  be  made  without  evaluating  node  6 or  any  of  its 
possible  descendants. 

In  Figure  2,  node  4 has  an  estimated  value  to  MAX  of  20.  When  node  6 is  evaluated  at 
26,  It  becomes  clear  that  MIN  should  avoid  moving  to  node  5.  Node  2 can  therefore  be 
assigned  a value  of  20  without  any  need  to  evaluate  node  7 or  any  of  its  descendants. 

The  alpha-beta  technique  for  evaluating  nodes  of  a game  tree  eliminates  these 
unnecessary  evaluations.  If,  as  is  usual,  the  generation  of  nodes  Is  interleaved  with  their 
evaluation,  then  nodes  such  as  the  descendants  of  node  5 in  Figure  1 and  of  node  7 in 
Figure  2 need  never  even  be  generated.  The  technique  uses  two  parameters,  alpha  and 
beta.  In  Figure  1,  the  parameter  alpha  carries  the  lower  bound  of  15  on  MAX'S  achievement 
from  node  1 ; the  elimination  of  node  6 Is  an  alpha  cutoff.  In  Figure  2,  the  parameter  beta  Is 
set  to  20  at  node  4,  representing  an  upper  bound  on  the  value  to  MAX  of  node  2;  the 
elimination  of  node  7 Is  a beta  cutoff.  The  procedure  guarantees  that  the  roct  node  of  the 
tree  will  have  the  same  final  value  as  if  exhaustive  mlnlmaxing  were  employed. 

A concise  statement  of  the  alpha-beta  procedure  has  been  given  by  Knuth  and  Moore 
(1976).  It  uses  their  negmax  representation  in  which  both  players  are  treated  as  wishing  to 
maximize  (see  Article  C5e).  Figure  3 shows  how  Figures  1 and  2 are  transformed  in  the 
negmax  representation. 
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Figure  3.  The  NEGMAX  representation  of  Figures  1 and  2. 

To  evaluate  node  1 of  either  tree,  the  procedure  is  called  with  the  parameters  POSITION  * 
node  1,  ALPHA  ■ negative  infinity,  and  BETA  ■ positive  infinity.  The  static  evaluation  function 
la  called  f.  The  procedure,  here  called  VALUE,  is  as  follows: 

INTEGER  PROCEDURE  value(POSITION  p,  INTEGER  alpha,  INTEGER  beta) 

BEGIN 

INTEGER  m,  I,  t.  d 

determine  the  successor  positions  pv  p2 pd 

of  position  p; 

IF  d ■ 0 THEN  value  :■  f(p)  ELSE 
BEGIN 
m :■  alpha; 

FOR  I :■  1 STEP  1 UNTIL  d DO 

BEGIN  t -value  (p,.  -beta,  -m); 

IF  t > m THEN  m :■  t; 

IF  m > beta  or  m ■ beta  THEN  GO  TO  done; 

END; 


done:  value  :■  m; 

END; 

END; 

For  an  Intuitively  developed  LISP  version  of  the  alpha-beta  procedure,  see  Winston  (1977). 
An  excellent  review  of  the  historical  development  of  the  technique  appears  In  Knuth  and 
Moore  (1976). 


Ordering  of  Successors 

The  degree  to  which  the  alpha-beta  procedure  represents  an  Improvement  in  efficiency 
over  straight  mlnlmaxing  varies  with  the  order  In  which  successor  nodes  are  evaluated.  For 
example,  no  cutoff  would  occur  In  Figure  1 if  node  3 were  considered  before  node  2. 


In  general,  It  Is  desirable  that  the  best  successor  of  each  node  be  the  first  one 
evaluated— that  Is,  that  the  first  move  MAX  considers  be  his  best  move,  and  that  the  first 


66 


Al  Handbook 


reply  considered  for  MIN  be  the  move  that  Is  best  for  MIN  and  worst  for  MAX.  Several 
schemes  for  ordering  the  successors  of  a node  have  been  described  to  try  to  achieve  this 
state  of  affairs.  One  possibility,  an  example  of  fixed  ordering , is  to  apply  the  static 
evaluation  function  to  the  successors,  taking  the  results  of  this  preliminary  evaluation  as  an 
approximation  of  their  expected  backed-up  values.  A method  of  this  sort  will  result  In  depth- 
first  generation  and  evaluation  of  the  partial  game  tree,  subject  to  the  depth  bound  or  other 
criteria  for  terminating  generation.  For  some  other  possibilities,  see  Article  C5c. 


Efficiency  in  Uniform  Game  Trees 

Since  the  alpha-beta  procedure  Is  more  complicated  than  minimaxing,  although  it  yields 
the  same  result,  one  may  inquire  how  great  an  increase  it  produces  in  search  efficiency. 
Most  theoretical  results  on  this  question  deal  with  uniform  game  trees:  A tree  is  said  to  be 
uniform  if  every  tip  node  has  depth  d and  every  nontip  node  has  exactly  b successors.  Here 
b is  called  the  branching  factor  or  degree  of  the  tree. 

The  results  reviewed  below  come  from  Knuth  and  Moore  (1976)  and.  for  the  Pest  case, 
Slagle  and  Dixon  (1360).  For  other  related  work,  see  Fuller  et  al.  (1073),  Newborn  (1077), 
and  Baudot  (1978). 

The  best  case.  A uniform  game  tree  of  depth  d and  degree  b contains  exactly  ba  tip 
nodes,  all  of  which  must  be  examined  by  minimax.  in  the  worst  case,  alpha-beta  also  must 
examine  every  tip  node.  In  the  best  case,  alpha-beta  examines  only  about  twice  the  square 
root  of  the  number  of  tip  nodes.  More  precisely,  assuming  the  value  of  the  root  is  not 
Infinite,  the  number  of  tip  nodes  examined  In  the  best  case  is 

♦ b[«/2]  - 1 

(where  square  brackets  represent  the  greatest  integer  function);  and  the  nodes  examined  In 
the  tree  as  a whole  are  precisely  the  critical  nodes,  defined  as  follows: 

Type  1 critical  nodes  are  the  root  node  and  all  first  successors  of  type  1 nodes. 

Type  2 critical  nodes  are  all  further  successors  (except  the  first)  of  type  1 
nodes  and  all  successors  of  type  3 nodes. 

Type  3 critical  nodes  are  the  first  successors  of  type  2 nodes. 

Figure  4 illustrates  the  distribution  of  critical  nodes  In  a uniform  tree  of  degree  3 and  depth 

3. 
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Figure  4.  Distribution  of  critical  nodes. 


Knuth  and  Moore  have  shown  that  the  best  case  occurs  for  a uniform  tree  if  the  best  move  is 
considered  first  at  each  critical  node  of  types  1 and  2.  Attempts  to  order  the  successors  of 
type  3 positions  contribute  nothing  to  efficiency,  since  these  successors  are  type  2 nodes, 
which  must  all  be  ■•x -mined  anyway. 

Random  uniform  game  trees.  Knuth  and  Moore  also  show  that  the  alpha-beta 
technique  is  optimal  in  the  sense  that  no  algorithm  can  evaluate  any  game  tree  by  examining 
fewer  nodes  than  alpha-beta  does  with  an  appropriate  ordering  of  successors.  Realistically, 
of  course,  one  cannot  expect  to  achieve  the  optimal  successor  ordering,  since  this  would 
imply  full  knowledge  of  the  game  tree  before  it  Is  generated.  Assuming,  therefore,  that  the 
tip  nodes  of  the  tree  have  distinct  random  values,  Knuth  and  Moore  show  that  the  expected 
number  of  tip  nodes  examined,  in  evaluation  of  a uniform  tree  with  branching  factor  b and 
depth  d,  has  an  asymptotic  upper  bound  of 

(b/(log  b))d 

as  d goes  to  infinity. 

Totally  dependent  uniform  game  trees.  One  other  type  of  tree  considered  by  Knuth 
and  Moore,  perhaps  more  realistic  than  the  one  in  which  tip  nodes  have  random  values, 
corresponds  to  games  In  which  each  move  is  critical:  If  a poor  move  Is  ever  chosen,  there  is 
no  way  to  recoup.  The  model  Is  a uniform  game  tree  that  is  totally  dependent:  For  any  two 
successors  of  node  p,  these  successors  can  be  labeled  q and  r so  that  every  tip  node 
descended  from  node  q has  greater  value  than  any  tip  node  descended  from  node  r.  In  this 
type  of  tree,  if  the  degree  is  at  least  3,  the  expected  number  of  tip  positions  examined  is 
bounded  by  a constant  (depending  on  the  degree)  multiplied  by  the  number  of  tip  nodes 
examined  by  the  alpha-beta  method  in  the  best  case. 

References 

See  Baudet  (197B).  Fuller,  Gaschnig  & Gillogly  (1973),  Knuth  & Moore  (1976),  Newborn 
(1977),  Nilsson  (1971),  Slagle  & Dixon  (1969),  Slagle  (1971),  and  Winston  (1977). 
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C6c.  Heuristics  In  Game  Tree  Search 

In  the  search  of  a game  tree  (Article  B3),  as  in  other  kinds  of  search,  there  are  various 
points  at  which  heuristic  information  may  be  applied.  The  parallel  Is  not  exact,  however.  In 
one-person  problem  solving,  the  main  uses  for  heuristic  information  are  to  decide  which  node 
to  expand  next,  which  operator  to  apply  next,  and,  in  some  algorithms,  which  nodes  to  prune 
from  the  search  tree.  (See  Article  C3a.)  In  game-playing  programs,  these  questions  also 
exist,  but  with  a shift  in  emphasis.  In  addition,  some  new  questions  arise:  When  should  the 
search  be  terminated?  How  should  a move  be  chosen  on  the  basis  of  the  search  that  has 
been  made? 

The  simplest  answers  to  these  questions  were  described  in  Article  C5a;  Expand  every 
node  completely,  In  any  convenient  order  and  with  no  pruning,  until  every  tip  node  represents 
a termination  of  the  game.  Then,  working  back  from  the  end  of  the  game,  use  the  minimax 
procedure  to  find  a winning  line  of  play  (if  one  exists),  and  follow  this  line  of  play  throughout 
the  game.  Article  C5b,  Alpha-beta  Pruning,  described  an  improvement  on  this  approach  that 
yields  the  same  final  result  with  greater  efficiency. 

A program  using  only  these  basic  techniques  would  play  a theoretically  perfect  game; 
its  task  would  be  like  searching  an  AND/OR  tree  for  a solution  to  a one-person  problem.  For  a 
simple  game  like  tic-tac-toe  (see  Article  B3),  such  a program  would  no  doubt  be  feasible. 
For  complex  games,  however,  it  has  been  recognized  from  the  beginning  that  searching  from 
the  start  of  the  game  to  its  end  would  be  impossible.  In  chess,  for  example,  with  around  30 
legal  moves  from  each  position  and  about  40  moves  for  each  player  in  a typical  game,  there 

are  some  (302)40  or  10,2°  different  plays  of  the  game  (Shannon,  1950). 

Because  of  the  magnitude  of  the  search  space  in  chess,  checkers,  and  other  nontrivial 
games,  there  is  a major  difference  between  programs  that  play  such  games  and  programs 
that  use  the  methods  of  this  chapter  to  solve  nonadversary  problems.  The  latter  either  find 
a solution  or  fall,  having  run  out  of  time  or  space;  much  of  the  research  assumes  that  some 
solution  can  be  found  and  deals  with  how  to  guarantee  that  it  is  optimal  or  nearly  optimal 
(see  Section  C3,  Heuristic  State-space  Search).  The  question  for  a chess  program,  in 
contrast,  Is  how  to  play  a good  game  even  though  it  has  not  found  a solution  to  the  problem 
of  winning.  Repeatedly  the  program  must  become  committed  to  its  next  move  long  before  the 
end  of  the  game  comes  into  view.  Whether  the  move  chosen  is  in  fact  part  of  a winning 
strategy  Is  unknown  until  later  In  the  game. 

For  a nontrivial  game  playing  program,  then,  the  Issues  listed  at  the  beginning  of  this 
article  are  all  aspects  of  a broader  question:  Can  the  basic  search  techniques,  designed  for 
seeking  a guaranteed  win,  be  successfully  adapted  to  the  problem  of  simply  choosing  the 
next  move?  In  addition,  one  might  well  ask  whether  there  are  alternatives  to  search  as  the 
basis  for  move  selection.  Most  of  the  work  exploring  these  questions  has  been  done  in  the 
specific  domain  of  chess.  In  general,  the  discussion  below  is  limited  to  chess  programs  and 
Samuel's  checkers  program  (1963,  1967). 


Alternatives  to  Search 

* An  example  of  choosing  a move  on  a basis  other  than  search  Is  the  use  of  "book 
moves"  In  the  opening  of  a chess  game  (see  Frey,  1077,  pp.  77-79).  More  generally,  there 
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is  an  emphasis  In  the  recent  computer  chess  literature  on  treating  the  problem  of  move 
choice  as  a problem  of  recognizing  patterns  on  the  board  and  associating  appropriate  playing 
methods  with  each  pattern  (o.g.,  Charness,  1977,  p.  62;  Bratko  et  al.,  1978;  Wilkins,  1979). 

It  Is  not  expected,  however,  that  search  can  be  eliminated  entirely  from  chess 
programs;  even  human  players  do  some  searching.  Rather,  the  choice-of-move  problem  is 
seen  as  Involving  a tradeoff  between  the  amount  of  specialized  chess  knowledge  a program 
has  and  the  amount  of  search  it  needs  to  do.  (See,  e.g.,  Berliner  1977c;  Michie,  1977.)  And 
there  are  limits  on  the  amount  of  knowledge  a program  can  be  given:  The  combinatorics  of 
chess  preclude  storing  an  exhaustive  representation  of  the  game;  and  even  the  knowledge 
possessed  by  chess  masters,  which  greatly  restricts  search  in  human  play,  also  remains  very 
far  from  complete  formalization. 

The  last  section  of  this  article  reviews  several  programs  that  attempt  to  use  human- 
like knowledge  to  eliminate  most  searching.  The  sections  preceding  it  concern  techniques 
used  in  programs  In  which  search  rather  than  knowledge  is  predominant. 


Search-based  Programs 

The  most  successful  game-playing  programs  so  far  have  made  search  rattier  than 
knowledge  their  main  Ingredient.  These  include,  among  the  earlier  programs,  Samuel’s 
checkers  program  (1963,  1967),  which  came  close  to  expert  play;  and  Greenblatt's  chess 
program  (1967),  which  was  the  first  to  compete  in  tournaments  and  which  earned  a rating  of 
1400-1460,  making  it  a Class  C player.  (Current  classes  of  the  United  States  Chess 
Federation  are  E through  A,  Expert,  Master,  and  Senior  Master.  See  Hearst,  1977,  p.  171.) 
Notable  later  programs  include  the  Soviet  program  KAISSA  (Adelson-Velskiy  et  al.,  1976), 
which  won  the  first  world  computer  chess  championship  in  1974,  and  Slate  and  Atkin's  CHESS 
4.6  (1977),  whose  current  standing  Is  mentioned  below.  (For  general  reviews  of  computer 
chess  competition,  see  Berliner,  1978a;  Mittman,  1977;  and  Newborn,  1976.) 

All  the  programs  referred  to  above  follow  the  basic  search  paradigm  formulated  by 
Shannon  In  1950.  In  its  simplest  form,  which  was  called  a Type  A program,  Shannon's 
paradigm  made  Just  two  changes  to  the  procedure  mentioned  above  that  calls  for  searching 
exhaustively  all  the  way  to  the  end  of  the  game.  First,  the  game  tree  was  to  be  generated 
only  to  a fixed  depth.  Second,  since  the  nodes  at  the  depth  limit  would  normally  be 
nonterminal,  a means  of  estimating  the  promise  of  these  nodes  was  required.  The  estimate 
was  to  be  given  by  a static  evaluation  function,  whose  values  could  then  be  backed  up  by 
minimaxing  to  determine  the  next  move.  After  this  move  was  made  and  the  opponent  had 
replied,  the  search  process  would  be  repeated  beginning  from  the  new  position. 

Shannon  noted  that  a simple  Type  A program  would  play  chess  both  badly  and  slowly. 
He  suggested  two  directions  for  improvement  in  a Type  A program,  with  which  the  program 
would  become  Type  B.  The  general  objectives  were,  first,  to  let  the  exploration  of  a line  of 
play  continue  to  a reasonable  stopping  point  instead  of  Invariably  cutting  It  off  at  an  arbitrary 
depth;  and,  second,  to  provide  some  selectivity  about  the  lines  of  play  considered,  so  that 
more  time  could  be  spent  investigating  strong  moves  and  less  on  pointless  ones. 


Even  a Type  B program,  Shannon  concluded,  seemed  to  rely  too  much  on  brute-force 
calculation  rather  than  on  knowledgeable  analysis  of  the  situation  to  choose  a move. 
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Nevertheless,  his  proposals  established  a framowork  that  most  competitive  game-playing 
programs  have  adopted.  7ho  framowork  raises  a largo  number  of  Interrelated  Issues,  which 
are  discussed  In  the  following  sections. 


Static  Evaluation 

A static  t valuation  function,  by  definition.  Is  ono  that  ostimaios  tho  value  of  a board 
position  without  looking  at  any  of  that  position's  successors.  An  idoal  function  would  bo  ono 
that  reports  whether  tho  position  loads  to  a~win,  a loss,  or  a draw  (provided  neither  side 
mnkos  a mistake).  Even  more  informatively,  tho  function  might  report  the  number  of  moves 
roquirod  to  win,  with  an  arbitrarily  largo  valuo  if  no  win  Is  posslblo.  But  functions  that  can 
distinguish  botwoen  winning  and  losing  positions  are  known  only  for  simple  games;  an 
example  of  such  a function  for  tho  gamo  Nim  is  givon  in  Shannon  (1950). 

Whore  porfoct  evaluation  functions  are  unavailable,  tho  actual  static  evaluator  must 
return  an  ostlmato.  Unlike  tho  evaluation  function  used  in  an  ordinary  stato-spaco  or  AND/OH 
graph  search  (C3n,  C4),  tho  static  evaluation  function  of  a game-playing  program  doos  not 
normally  attempt  directly  to  ostlmato  tho  distance  to  a win  from  tho  position  evaluated.  (For 
a proposal  that  tho  function  should  do  Just  this,  soo  Harris,  1974.)  Instead,  the  function  Is 
usually  a linear  polynomial  whoso  terms  represent  various  features  of  the  position,  high 
values  being  given  for  features  favorable  to  tho  program  and  low  ones  for  those  favoring  tho 
opponent.  In  chess,  the  most  Important  feature  Is  material,  the  relative  value  of  each  side's 
pieces  on  tho  board.  Othor  typical  foaturos,  familiar  to  choss  players,  include  king  safety, 
mobility,  center  control,  and  pawn  structure. 

The  most  oxtondod  troatmont  of  evaluation  functions  In  tho  literature  Is  provided  by 
Samuel  (1903,  1907).  For  chcckors,  ho  concluded  (1967,  p.  01  1)  that  tho  optimal  number  of 
foaturos  to  bo  usod  in  tho  evaluation  function  was  between  twenty  and  thirty.  Samuel's  main 
Intorost  was  In  machino  learning;  ono  approach  ho  took  was  to  provide  his  chockers  program 
with  a large  sot  of  foaturos  for  possible  uso  In  tho  evaluation  function  and  to  lot  tho  program 
determine,  as  it  gained  playing  oxporienco,  both  which  of  those  foaturos  should  bo  Included 
and  what  their  relative  weights  should  bo.  In  a later  version  of  the  program,  tho  emphasis 
was  shifted  to  taking  tho  Interactions  among  foaturos  into  account  in  evaluating  positions. 
With  this  change,  tho  evaluation  function  bocamo  nonlinear,  and  considerable  Improvement 
was  reportod  In  Its  quality  as  measurod  by  tho  correlation  with  movos  chosen  In  master  play 
(Samuel,  1967;  see  also  Griffith,  1974).  For  furthor  discussion  of  Samuel's  work,  soo 
Learning. 

Reasonably  accurate  static  evaluation,  thon,  requires  a rather  complex  function.  But 
there  Is  an  Important  limit  on  tho  complexity  that  Is  foasiblo,  especially  for  a program  that 
plays  In  tournaments,  under  tlmo  limitations.  As  the  total  number  of  tip  nodes  in  tho  search 
tree  increases,  tho  time  avallablo  for  evaluating  any  singlo  tip  node  goes  down.  Thus 
Glllogly's  choss  program  TECH  (1972),  which  was  Intondod  as  an  experiment  In  how  much 
could  be  accomplished  on  advanced  machines  by  simple  brute  force  search,  and  which 
generates  up  to  600,000  tip  nodes  even  with  alpha-beta  pruning,  uses  material  as  the  only 
factor  in  Its  static  evaluations. 
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Backed-up  Evaluation 

The  Shannon  paradigm  assumes  that  the  step  between  static  evaluation  and  the  choice 
of  a move  Is  simply  mlnimaxing:  The  program  moves  to  any  position  with  the  best  backed-up 
mlnimax  value.  This  step  Is  Indeed  very  commonly  usod.  But  it  Is  worth  noting  that,  since  the 
static  evaluation  function  may  be  wrong,  the  minlmax  procedure  no  longer  serves  its  original 
purpose  of  defining  and  identifying  a move  that  is  theoretically  correct.  Instead,  minimaxing 
has  itself  become  a heuristic  for  the  choice  of  move.  Several  programs  have  therefore 
experimented  with  varying  or  supplementing  the  minimax  procedure.  Slagle  and  Dixon 
(1970),  for  example,  in  experiments  with  the  game  of  kalah,  compute  the  backed-up  value  of 
a node  by  taking  into  account  not  only  the  value  of  its  best  successor  but  also  whether  the 
node  has  several  good  successors  or  just  one.  Gillogly's  TECH  (1972),  having  computed 
minimax  values  on  the  basis  of  an  extremely  simple  static  evaluation,  breaks  ties  between 
moves  with  equal  minlmax  values  by  an  analysis  of  features  not  considered  by  the  evaluation 
function.  Newell,  Shaw,  and  Simon  (1963a)  set  a value  In  advance  that  the  search  is 
expected  to  achieve;  the  first  move  found  that  meets  this  standard  is  made,  and  only  if  no 
move  Is  good  enough  Is  the  bost  mlnimax  value  used  to  determine  the  move  (see  also  Newell 
& Simon,  1972). 


Depth  of  Search 

If  perfect  evaluation  functions  were  available,  a game-playing  program  could  proceed 
at  each  turn  by  generating  all  legal  moves,  evaluating  each  of  the  resulting  positions,  and 
choosing  the  move  leading  to  the  best  value.  The  reason  for  looking  farther  ahead  is  to 
compensate  for  errors  in  the  static  evaluation.  The  assumption  is  that,  since  static 
evaluation  has  a predictive  aspect,  there  will  be  less  room  for  mistaken  prediction  If  a deep 
tree  Is  generated  before  the  evaluation  function  is  applied. 

The  controlling  fact  about  search  depth  Is  the  combinatorial  explosion.  If  the  average 
number  of  legal  moves  from  a position,  the  branching  factor,  is  b,  the  game  tree  will  have  about 

bd  nodes  at  depth  d.  According  to  Shannon's  estimate  for  chess,  a complete  tree  carried  to 
depth  6--3  moves  for  each  player--would  already  have  about  one  billion  tip  nodes.  At  the 
same  time,  Shannon  noted,  a world  champion  may  occasionally  look  ahead,  along  a single  line 
of  play,  to  a depth  as  great  as  15  or  20.  More  recently  Hans  Berliner,  a former  World 
Correspondence  Chess  Champion,  has  said  he  finds  It  necessary  at  least  once  in  a game  to 
look  ahead  to  a depth  of  14  or  more  (1974,  p.  1-8).  The  question,  then,  Is  how  to  get  the 
needed  depth,  In  the  right  places,  without  succumbing  to  the  combinatorial  explosion.  An 
alternative  question  would  be  how  to  avoid  the  need  for  so  deep  a search.  The  remainder  of 
this  article  concerns  attempts  to  solve  or  at  least  alleviate  these  problems.  First,  however, 
experience  with  the  use  of  depth  bounds  as  such  will  be  reviewed. 

Fixed-depth  search  with  extensions  for  quiescence.  The  simplest  lookahead 
procedure,  which  was  called  for  by  Shannon's  Type  A strategy,  Is  to  set  a fixed  depth,  or 
ply,  to  which  the  game  tree  Is  to  be  generated  and  to  apply  the  static  evaluation  function 
only  to  nodes  at  this  depth.  Thus  a 4-ply  search  would  statically  evaluate  the  positions 
reached  after  exactly  two  turns  for  each  player.  There  are  serious  drawbacks  In  this 
procedure,  as  Shannon  observed,  and  It  was  usod  only  in  very  early  programs  (Kister  et  al., 
1967;  Bernstein  et  al.,  1059).  For  example,  a chess  evaluation  function  based  mainly  on 
material  cannot  return  a realistic  value  if  at  the  depth  limit  the  players  happen  to  be  halfway 
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through  an  exchange  of  pieces.  The  concept  of  a quiescent  or  deed  position  was  introduced  to 
get  around  such  difficulties  (Shannon,  1050;  see  also  Turing,  1953):  Soarch  would  be 
extended  beyond  the  normal  limit,  from  nonciuioscont  positions  only,  until  all  tip  nodes  were 
relatively  stable  or  perhaps  until  some  absolute  depth-bound  had  boon  reached. 

This  introduction  of  a quiescence  search  was  one  of  the  two  features  that  changed  a 
program,  In  Shannon's  terminology,  from  Typo  A to  Typo  B.  On  Shannon's  suggested 
definition,  a position  was  considered  nonquioscont  if  "any  piece  Is  attacked  by  a piece  of 
lower  value,  or  by  more  pieces  than  defences  or  if  any  chock  exists  on  a square  controlled 
by  opponent"  (1950,  p.  271).  Many  programs  havo  adopted  a similar  definition,  with  the 
result  that  the  only  moves  examined  beyond  the  normal  limit  are  chocks  and  immediate 
captures  (e.g.,  Glllogly,  1972;  Adelson-Volskiy  et  al.,  1975;  Slate  & Atkin,  1977).  If  such  a 
quiescence  search  is  combined  with  considering  all  legal  moves  down  to  the  normal  depth 
limit,  the  program  Is  still  called  Typo  A in  current  terminology  (e.g.,  Berliner,  1978a). 

The  horizon  effect.  Searching  to  an  arbitrarily  limited  depth,  oven  with  extensions  for 
checks  and  captures,  creates  a phenomenon  that  Berliner  (1073,  1974)  has  called  the 
horizon  effect.  Berliner's  general  observation  is  that,  whenever  search  Is  terminated  (short  of 
the  end  of  the  gamo)  and  a static  evaluation  function  is  applied,  the  program's  "reality 
exists  in  terms  of  the  output  of  the  static  evaluation  function,  and  anything  that  is  not 
detectable  at  evaluation  timo  doos  not  exist  as  far  as  the  program  Is  concerned"  (1974,  p. 
1-1). 


Two  kinds  of  orrors  ensuo.  The  first  is  called  the  negative  horizon  effect:  The  program 
manipulates  the  timing  of  moves  to  force  certain  positions  to  appear  at  t he  search  horizon, 
and  It  thus  may  conclude  that  it  has  avoided  some  undcsirablo  effect  when  in  fact  the  effect 
has  only  been  delayed  to  a point  beyond  the  horizon.  A second  kind  of  error,  the  positive 
horizon  effect,  Involves  reaching  for  a desirable  consequence:  Either  the  program  wrongly 
concludes  that  the  consequence  is  achievable,  or  it  fails  to  realize  that  the  same 
consequence  could  also  be  achieved  later  In  the  gamo  in  a more  effective  form.  This  last 
problem,  Berliner  believes,  can  bo  met  only  by  finding  ways  to  represent  and  use  more  chess 
knowledge  than  traditional  programs  have  Included  (1074,  p.  1-7). 

For  most  of  the  errors  coming  from  the  horizon  effect,  however,  the  diagnosis  is  that 
the  typical  definitions  of  quioscencc  are  highly  oversimplified.  Ideally  a position  would  bo 
considered  quiescent  only  when  the  static  evaluation  function,  applied  to  that  position,  could 
return  a realistic  value,  that  is,  when  tho  value  of  every  term  included  in  the  function  had 
become  stable.  A quiescence  search  that  pursues  only  captures  and  checking  moves, 
however,  considers  only  changes  in  the  material  term.  Tho  material  term  itself,  moreover, 
usually  reflects  only  tho  presence  of  tiro  pieces  on  tho  board;  its  value  will  bo  unchanged  by 
a move  that  guarantees  a capture  lator  instead  of  making  a capture  now. 

To  get  around  the  problems  arising  from  inadequate  quiescence  analysis,  a first 
approach  called  secondary  search  was  dovoloped  by  Groonblatt  (1067):  Whenever  a move 
appeared,  on  the  basis  of  the  regular  search  (including  quiescence),  to  be  the  best  move 
considered  so  far,  the  predicted  lino  of  play  wus  oxtonded  by  searching  another  two  ply 
(plus  quiescence)  to  test  the  evaluation.  Berliner  points  out,  however:  "Tho  horizon  effect 
cannot  be  dealt  with  adequately  by  morely  shifting  tho  horizon"  (1974,  p.  1-4).  One  direction 
In  current  work,  therefore,  looks  toward  a much  fuller  quioscence  analysis  as  a substitute  for 
arbitrary  depth  bounds.  (See  Harris,  1975,  1977;  Slate  & Atkin,  1977,  pp.  116-117;  and, 
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for  an  early  example.  Newell  & Simon,  1972,  pp.  678-698.)  Berliner  meanwhile  is  developing 
a general  algorithm,  not  limited  to  chess,  for  causing  tree  search  to  terminate  with  a best 
move,  even  though  no  depth  limit  has  been  set  and  no  full  path  to  a win  has  been  found 
(Berliner,  1977c,  1978b). 

Iterative  deepening.  Despite  its  drawbacks,  most  current  programs  still  use  a fixed- 
depth  search,  extended  for  checks  and  capture  sequences.  A variation  used  by  CHESS  4.5 
(Slate  «.  Atkin,  1977)  Is  called  iterative  deepening : A complete  search,  Investigating  all  legal 
moves  (subject  to  alpha-beta  pruning),  Is  done  to  depth  2,  returning  a move.  The  search  is 
then  redone  to  depth  3,  again  to  depth  4,  and  so  on  until  a preset  time  limit  is  exceeded.  For 
efficiency,  Information  from  earlier  Iterations  is  saved  for  use  In  later  ones.  Running  on  the 
very  fast  CDC  Cyber  176,  the  program  searches  to  an  average  depth  of  6 plies  in 
tournament  play,  with  search  trees  averaging  600,000  nodes  (Newborn,  1978).  It  is  the 
first  program  to  have  achieved  an  Expert  rating  in  human  play.  In  the  fall  of  1978  a new 
version,  CHESS  4.7,  was  reportedly  rated  2160  (Levy,  1979);  Master  ratings  begin  at  2200. 
It  remains  an  open  question  how  much  stronger  the  program  can  become. 


Ordering  of  Search 

The  Shannon  paradigm  did  not  specify  any  particular  order  In  which  the  nodes  of  the 
search  tree  were  to  be  explored  or  in  which  moves  from  a given  node  were  to  be  considered. 
For  efficient  use  of  space,  the  order  of  node  expansion  is  usually  depth-first;  a depth-first 
algorithm  needs  to  store  explicitly  only  those  nodes  on  the  path  It  is  currently  Investigating 
and  not  the  parts  of  the  tree  where  search  has  been  completed. 

With  the  Invention  of  alpha-beta  pruning,  the  order  of  considering  moves  within  a 
depth-first  search  became  highly  significant.  If  the  order  Is  Ideal,  then  in  a tree  with 
branching  factor  b the  number  of  nodes  that  must  be  examined  at  depth  d Is  reduced  from  bd 
to  only  about  2bd/2.  (See  Article  C5b.)  For  example,  Shannon's  estimated  109  chess 
positions  at  depth  6 would  be  reduced  to  around  50,000.  It  also  follows  that,  for  a constant 
number  of  tip  nodes  examined,  correct  ordering  of  the  moves  for  alpha-beta  cutoffs  would 
allow  the  search  depth  to  be  roughly  doubled.  In  general,  the  desired  ordering  is  one  in 
which  the  first  move  considered  at  a position  is  the  best  move  for  the  player  whose  turn  It  is. 
Usually,  of  course,  there  is  no  method  guaranteed  to  achieve  this  ordering,  for  If  there  were, 
It  would  enable  moves  to  be  chosen  with  no  search  at  all.  Several  heuristics  have  been 
used,  however,  to  try  to  approximate  optimal  ordering. 

Perhaps  the  simplest  Idea  for  move  ordering  Is  the  fixed-ordering  method  mentioned  In 
Article  C5b:  For  each  move  from  a node,  generate  a new  node  for  the  resulting  position, 
apply  the  static  evaluation  function  to  the  position,  and  order  the  nodes  according  to  this 
preliminary  estimate.  For  greater  efficiency,  several  programs  have  used  a separate 
function  for  move  ordering,  which  applies  to  the  move  Itself  instead  of  to  the  position  that 
results  from  It  (Greenblatt,  1967;  Berliner,  1974,  p.  11-16;  Adelson-Velskiy,  1976).  In  either 
case  the  game  tree  Is  explored  by  an  ordered  depth-first  search  (Article  C3a). 

A fuller  basis  for  choosing  which  move  to  consider  first  is  provided  by  Slate  and  Atkin's 
Iterative  deepening  technique,  which  makes  repeated  depth-first  searches.  Each  Iteration 
constructs  a line  of  play,  down  to  its  depth  limit,  consisting  of  apparently  best  moves.  The 
following  Iteration,  going  one  ply  deeper,  thus  has  available  an  estimated  best  move  from 
each  position  along  this  line  of  play.  (See  Slate  & Atkin,  1977,  pp.  102-103.) 
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A furthor  approach  to  move  ordering  makes  explicit  the  Idea  of  a refutation  move:  For 
each  move  that  is  not  a best  move,  it  should  bo  shown  as  quickly  as  possible  that  the  move 
is  bad.  To  do  this,  strong  replies  should  bo  considered  first,  which  may  refute  the  move 
proposed.  Typical  implementations  consider  all  capturing  moves  first,  and  then  consider  killer 
moves.  The  idea  here,  called  the  killer  heuristic,  is  that  if  a move  has  served  as  a refutation  in 
some  previously  examined  position  that  Is  similar  to  the  current  one,  It  Is  likely  to  be  a 
refutation  In  the  current  position  too.  For  more  on  the  killor  heuristic  and  other  refutation 
techniques,  see  Slate  and  Atkin  (1977),  Adelson-Velskiy  (1976),  Gillogly  (1972),  and  Frey 
(1977). 

Once  the  moves  have  boen  ordered  at  a given  node  and  the  search  has  moved 
downward,  following  the  move  that  seemed  best,  it  may  turn  out  that  this  move  Is  actually  a 
very  bad  one  for  reasons  that  wore  not  apparent  earlier.  Since  accurate  move  ordering  is 
important  to  maximizing  alpha-beta  cutoffs,  it  might  be  worthwhile  at  this  point  to  go  back, 
reorder  the  moves,  and  start  again  with  a different  estimated  best  move.  Such  a procedure, 
called  dynamic  ordering,  was  Investigated  by  Slagle  and  Dixon  (1969),  using  the  gnme  ot 
kalah.  They  reported  a modest  improvement  ovor  fixod  ordering  for  trees  of  depth  at  h ast 
6.  On  the  other  hand,  Berliner's  chess  program  experienced  a serious  Increase  r r"  !ng 
time  when  dynamic  ordering  was  used  (1974,  p.  IV-14).  A procedure  somewhat  o 

dynamic  ordering  was  also  used  by  Samuel  (1967). 

If  dynamic  ordering  Is  carried  to  its  limit,  so  that  reordering  is  considered  ever>  » a 
node  is  expanded  Instead  of  only  under  more  limited  conditions,  the  search  procei.  e in 
effect  changes  from  depth-first  to  best-first.  That  is,  the  move  considered  next  (or  the 
position  to  which  It  leads)  is  on  some  estimate  the  most  promising  in  the  entire  search  tree 
generated  so  far,  subject  to  whatever  depth  limit  exists.  Nilsson  (1968,  1971)  implements 
this  Idea  by  adapting  his  algorithm  for  best-first  AND/OR  tree  search  (C4)  to  game  trees. 
Harris  (1076,  1977)  suggests  another  adaptation,  In  which  the  motivation  of  maximizing 
alpha-beta  pruning  no  longer  plays  a role  and  instead  the  objective  is  to  expand  the  most 
active  positions  first,  using  a thorough  quiescence  analysis  rather  than  a depth  limit  as  the 
criterion  for  search  termination. 


Width  of  Search 

The  techniques  discussed  so  far  are  consistent  with  the  idea  that  all  legal  moves  from 
a position  must  bo  examined,  at  least  sufficiently  to  establish  that  they  can  be  safely  pruned 
by  alpha-beta.  This  consideration  of  all  legal  moves  is  referred  to  as  full-width  searching. 
Some  of  the  earliest  programs  used  a full-width  search  for  simplicity;  strong  current  programs 
use  It  because  of  the  great  difficulty  in  determining,  without  search,  which  moves  can  be 
safely  Ignored  (Turing,  1953;  Kistcr  et  al.,  1957;  Gillogly,  1972;  Adelson-Velskiy  et  al., 
1976;  Slate  8.  Atkin,  1977).  The  problem,  of  course,  is  that  an  excellent  move  may  look  very 
poor  at  first  sight. 

Yet  the  average  number  of  legal  moves  from  a chess  position  Is  at  least  30,  and  even 
with  a maximum  of  alpha-beta  pruning  the  troe  grows  exponentially.  Making  the  search  more 
selective  was  Shannon's  second  requirement  to  change  a program  from  Type  A to  Type  B. 
Many  people  have  been  convinced  that  such  selectivity  Is  essential  to  a strong  chess 
program,  both  In  order  to  Increase  search  depth  and  to  permit  more  sophisticated  evaluation 
of  the  nodes  remaining  In  the  search  tree.  Berliner,  for  example,  has  advocated  reducing  the 


total  search  tree  size  to  at  most  6000  nodes,  with  a branching  factor  of  less  than  1 .9 
(1974,  p.  1-16).  Although  some  reconsideration  of  these  ideas  has  been  prompted  by  the 
success  of  CHESS  4.7  using  full-width  search,  It  appears  that  that  program  Is  still  weak  at 
long  endgame  sequences  (see  Berliner,  1978a;  Michie  & Bratko,  1978).  Moreover,  there  are 
other  games  for  which  It  is  even  clearer  that  full-width  search  is  not  the  answer.  For  the 
game  of  go,  for  example,  the  average  branching  factor  has  been  estimated  at  perhaps  200 
(Thorp  & Walden,  ’C70),  and  for  backgammon,  where  legal  moves  depend  on  the  throw  of  the 
dice  as  well  as  the  board  position,  the  factor  Is  over  800  (Berliner,  1977a). 

Various  devices  nave  bean  tried  in  the  effort  to  increase  the  selectivity  of  the  search 
without  missing  good  moves.  Somo  are  conceptually  simple,  Introducing  little  or  no  new 
chess-specific  knowledge  Into  the  program.  Others  attempt  to  formulate  and  use  chess 
concepts  as  sophisticated  as  thosi  a chess  master  might  employ.  The  remainder  of  this 
section  reviews  mainly  the  earlier  soarth-controlling  devices.  The  following  section  mentions 
worx,  some  of  which  moves  outside  the  Shannon  paradigm,  In  which  the  effort  to  capture 
expert  chess  knowledge  becomes  primary. 

Forward  pruning.  One  way  of  limiting  the  number  of  moves  to  be  considered  introduces 
no  new  complications;  Simply  generate  all  legal  moves  at  a position,  use  a fixed-ordering 
scheme  to  sort  them  according  to  thoir  apparent  goodness,  or  plausibility,  and  then  discard  all 
but  the  best  fow  moves.  Such  a technique,  called  plausible-move  generation  or  forward 
pruning,  was  used  by  Kotok  (1962)  and  Greenblatt  (1967);  see  also  Samuel  (1967).  A 
further  feature  of  those  programs,  sometimes  called  tapered  forward  pruning,  was  that  the 
number  of  moves  retained  was  a function  of  the  depth  at  which  they  were  generated.  For 
example,  Greenblatt's  program  in  tournament  play  retained  15  moves  from  a position  at  either 
of  the  top  two  levels  of  the  treo,  9 moves  at  the  next  two  levels,  and  7 moves  thereafter. 
These  figures  could  be  increased  in  special  cases--for  example,  to  be  sure  that  moves  of 
more  than  a single  piece  were  considered. 

Another  form  of  forward  pruning,  distinct  from  plausible  move  generation,  operates  not 
at  the  time  when  moves  are  originally  generated  but  later,  when  one  of  these  moves  (or  the 
position  to  which  It  leads)  Is  being  selected  for  further  exploration.  At  this  point  a preliminary 
estimate  of  the  value  of  tho  move  or  position  may  alroady  have  been  made  by  the  move- 
ordering scheme.  If  this  estimate  is  outside  the  limits  alpha  and  beta,  the  currently  known 
bounds  on  the  outcome  of  the  entire  search  (see  C5b),  the  node  is  pruned  without  further 
Investigation.  It  Is  possible,  of  course,  that  the  actual  backed-up  value  of  the  node  would 
have  turned  out  to  be  between  alpha  and  beta.  In  that  case  a good  move  may  have  been 
missed.  (See  Samuel,  1967;  Berliner,  1974,  p.  IV-13.) 

Still  another  basis  for  forward  pruning  has  been  explored  by  Adelson-Velskiy  et  al. 
(1976).  They  observe  that  KAISSA's  search  trees  include  many  lines  of  play  that  a human 
would  consider  absurd,  not  necessarily  because  the  moves  are  bad  a priori,  but  because  the 
human  player  has  already  considered  and  rejected  the  same  moves  In  an  analogous  position. 
The  proposal,  then,  is  to  remember  moves  that  have  been  found  to  be  absurd  (on  some 
definition)  and  to  reject  them  in  other  positions  too  unless  there  has  been  an  appropriate 
change  of  circumstances.  In  effect,  this  method  of  analogies  Involves  trying  to  establish 
conditions  under  which  a refutation  is  guaranteed  to  be  effective.  Then  the  line  of  play 
constituting  the  refutation  would  not  need  to  be  explored  separately  every  time  it  Is 
applicable.  (See  Frey,  1977,  p.  68.) 
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Goal-directed  move  generation.  Returning  to  the  initial  generation  of  moves,  there  is 
another  kind  of  plausible  move  generator  that  comes  closer  to  mimicking  tho  way  that  humans 
might  decide  which  moves  are  worth  considering.  Instead  of  generating  all  legal  moves  and 
discarding  some,  this  approach  does  not  generate  moves  at  all  unless  they  seem  relevant  to 
some  goal.  The  earliest  step  in  this  direction  was  Bernstein's  program  (1959),  which 
contained  a sequence  of  board  featuit  s to  be  tested  for  and  a procedure  for  generating 


moves  In  response  to  each  feature  that  was  present.  The  first  few  tests  tn  the  sequence 
were  (1)  Is  the  king  In  check?  (2)  can  material  be  gained,  lost,  or  exchanged?  and  (3)  is 
castling  possible?  A maximum  of  7 plausible  moves  was  returned.  Questions  later  In  the 
sequence  were  not  asked  if  earlier  questions  caused  the  maximum  to  be  reached.  Searching 
to  a fixed  depth  of  4 ply,  the  program  gonerated  trees  with  about  2400  tip  nodes. 

More  explicitly  goal-directed  move  generation  was  included  in  Newell,  Shaw,  and 
Simon's  1958  chess  program  (Newell,  Shaw,  & Simon,  1963a;  Newell  & Simon,  1972). 
Indeed,  the  entire  program  was  organized  in  terms  of  goals,  although  only  three--material, 
center  control,  and  piece  developmcnt--were  actually  Implemented.  At  each  turn,  the 
program  began  by  making  a preliminary  analysis  to  decide  which  of  the  goals  were  relevant 
to  the  situation;  these  were  entered,  in  order  of  importance,  on  a current  goal-list.  It  was 
Intended,  In  a more  fully  developed  program,  that  as  the  game  progressed  the  goals  of  center 
control  and  development  would  drop  out,  since  they  are  important  mainly  in  the  opening,  and 
would  be  replaced  by  others  more  appropriate  to  later  phases  of  the  game. 

Each  active  goal  in  the  Newell,  Shaw,  and  Simon  program  was  responsible  for 
generating  relevant  moves  at  the  first  level  of  the  tree.  In  addition,  each  goal  contained  its 
own  separate  generator  for  moves  at  deeper  levels,  its  own  criteria  for  whether  a position 
was  dead,  and  its  own  static  evaluation  function.  The  search  proceeded,  In  a highly  selective 
manner,  until  the  tip  nodes  were  dead  with  respect  to  all  active  goals.  Static  evaluations 
with  respect  to  the  various  goals  were  combined  lexicographically,  so  that  the  highest 
priority  goal  was  dominant  and  the  others  served  only  as  tiebreakers.  Newell  and  Simon 
report  that  the  program's  average  search  tree  contained  only  13  nodes--with  no  apparent 
loss  In  playing  power  compared  to  other  programs  up  to  that  time  (1972,  p.  694). 


Knowledge-based  Programs 

The  Bernstein  and  Newell,  Shaw,  and  Simon  programs  were  early  efforts  to  introduce 
significant  chess  knowledge,  organized  in  human  terms,  to  limit  brute-force  search.  The 
actual  knowledge  was  very  sketchy;  apparently  neither  program  ever  won  a game  (see 
Newell  & Simon,  1972,  pp.  677,  690). 

An  attempt  at  fuller  use  of  choss  knowledge  was  made  in  Berliner's  program.  CAPS-II 
(1974,  1977b).  Much  of  the  work  involved  developing  a representation  suitable  for  use  in 
selectively  generating  moves,  making  preliminary  evaluations  of  the  moves  so  proposed,  and 
describing  the  actual  consequences  discovered  when  a move  was  tried.  The  moves 
generated  depend  on  the  current  goal  state,  which  may  be  King  in  Check,  Aggressive, 
Preventive  Defense,  Nominal  Defense,  Dynamic  Defense,  or  Strategy.  In  contrast  to  the 
Newell,  Shaw,  and  Simon  program,  the  goal  states  are  mutually  exclusive,  and  state 
transitions  occur  dynamically  as  the  tree  is  searched,  in  accordance  with  a complex 
flowchart.  An  Important  feature  of  the  program,  the  Causality  Facility,  relates  to  both  move 
generation  and  move  ordering,  as  well  as  to  pruning  in  some  cases.  The  problem  it  attacks  is 
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a general  one  In  tree  searching:  When  a path  has  been  explored  and  found  unsatisfactory, 
most  programs  have  no  way  to  diagnose  what  went  wrong  and  use  this  Information  In  deciding 
where  to  search  next. 

The  basic  search  algorithm  in  CAPS-ll  is  depth-first,  with  minimaxing  and  alpha-beta 
pruning.  The  Causality  Facility  operates  as  a refinement  on  this  search.  A first  new  feature  is 
that,  whenever  a value  Is  backed  up  in  the  search  tree  as  a tentative  minimax  value,  certain 
information  is  accumulated  about  the  consequences  of  the  move  or  moves  that  produced  the 
value.  The  data  structure  in  which  the  Information  Is  stored  is  called  a Refutation  Description. 
As  the  basis  for  making  use  of  the  Refutation  Description,  the  program  uses  a variable 
representing  the  expected  value  of  the  position  at  the  root  of  the  search  tree;  this  value, 
which  may  be  updated  during  the  search,  lies  somewhere  between  the  bounds  given  by  alpha 
and  beta.  Now,  the  tentative  value  newly  backed  up  to  a node  can  be  compared  with  the 
expected  value.  If  the  comparison  is  unsatisfactory,  the  Causality  Facility  uses  the 
Refutation  Description  to  decide  whether  the  last  move  tried  from  the  node  could  have  been 
responsible,  it  generates  a list  of  alternative  moves  from  the  node,  with  the  aim  of  avoiding 
the  unsatisfactory  result.  These  moves  are  compared  with  the  list  of  moves  from  the  node 
that  had  been  generated  earlier  but  which  have  not  yet  been  tried.  The  comparison  is  used 
to  reorder  moves  already  on  the  untried  list  and,  depending  on  the  state  the  program  is  in,  to 
add  new  moves  to  the  list  and  to  prune  old  ones. 

Whereas  Berliner's  program  plays  the  full  game  of  chess,  there  are  several  other 
recent  programs  which,  in  their  emphasis  on  representing  chess  knowledge,  limit  their  task  to 
solving  problems  that  involve  only  selected  aspects  of  the  game.  Two  of  these  are  the 
programs  of  Pltrat  (1977)  and  Wilkins  (1979).  In  each,  the  task  Is  to  find  a line  of  play  that 
wins  material,  beginning  from  a given  middle-game  position.  The  approach  in  both  programs  is 
to  work  backward  from  the  goal  of  winning  material  to  a structure  of  subgoals  that 
constitutes  a plan.  (See  Planning.)  Wilkins's  program,  PARADISE,  for  example,  has  as  a main 
theme  the  expression  of  chess  concepts,  like  making  a square  safe  for  a piece  or  safely 
capturing  a piece,  in  terms  that  can  be  used  as  subgoals  and  eventually  reduced  to  specific 
moves.  Initially,  a plan  is  based  not  on  search  but  on  an  extensive  analysis  of  the  originally 
given  position;  it  may  contain  conditional  branches  depending  on  general  categories  of  moves 
with  which  the  opponent  might  reply.  The  general  plan  is  then  used  to  guide  search, 
generating  a very  small  tree.  Moves  considered  for  the  program  to  make  are  only  those 
relevant  to  the  current  subgoal;  for  the  simulated  opponent,  all  reasonable  defensive  moves 
are  considered,  if  search  shows  that  the  plan  has  failed,  a causality  facility  similar  to 
Berliner's  is  used  to  analyze  the  difficulty  and  suggest  a new  plan. 

Both  the  Pltrat  and  the  Wilkins  programs  have  succeeded  in  solving  problems  where  the 
winning  line  of  play  goes  to  a depth  of  around  20  ply.  Pitrat  reports,  for  a set  of  1 1 
problems,  that  search  tree  sizes  ranged  from  about  200  to  22,000  nodes;  computation  time 
varied  from  under  3 seconds  to  about  7.5  minutes.  Wilkins's  PARADISE  generates  smaller 
trees  but  uses  more  time;  for  89  problems  solved,  the  number  of  nodes  in  the  search  tree  ran 
from  a minimum  of  3 to  a maximum  of  215,  and  time  to  find  the  solution  varied  from  19 
seconds  to  33  minutes.  Wilkins  also  reports  a good  success  rate  compared  to  previous 
programs  tested  on  the  same  set  of  problems,  including  Berliner's  program,  Gillogly's  TECH, 
and  an  earlier  version  of  CHESS  4.5.  The  programs  other  than  PARADISE,  however,  were 
tested  with  a time  limit  of  only  5 minutes  per  problem. 

A final  example  of  the  use  of  chess  knowledge  to  solve  a class  of  problems  Is  the  work 
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of  Donald  Michie  and  his  colleagues  on  chess  endgames  (e.g.,  Bratko,  Kopec,  & Michie,  1978; 
Michie  & Bratko,  1978).  Here  each  combination  of  pieces  with  which  the  endgame  may  be 
played  Is  treated  as  posing  a separate  problem.  One  problem,  denoted  KNKR,  is  to  defend 
with  king  and  knight  against  king  and  rook,  starting  from  any  of  some  3 million  legal  positions 
involving  only  those  pieces.  The  objective  is  to  provide  the  program  with  enough  knowledge 
about  this  specific  class  of  chess  problems  to  achieve  theoretically  correct  play,  even  in 
situations  where  chess  masters  sometimes  err,  and  to  accomplish  this  using  only  a moderate 
amount  of  search. 

The  program's  knowledge  is  encoded  in  a data  structure  called  an  Advice  Table,  within 
which  patterns  occurring  on  the  board  may  be  described.  Each  pattern  has  an  associated 
list  of  goals,  or  "pieces  of  advice,"  in  the  order  in  which  they  should  be  attempted.  The 
object  then  becomes  to  find  a solution--in  the  sense  of  a solution  subtree  of  an  AND/OR  tree 
(C2)--to  the  problem  of  satisfying  one  of  the  goals.  Unlike  a standard  AND/OR  tree  search, 
however,  the  "advice"  includes  not  only  a definition  of  when  tip  nodes  should  be  considered 
terminal,  but  also  constraints  that  every  Intermediate  node  in  the  solution  tree  must  satisfy. 

The  amount  of  search  required  to  find  a solution  using  an  Advice  Table  depends  on  how 
much  knowledge  the  table  contains.  If  the  only  goal  provided  were  avoidance  of  mate,  a 
search  to  the  impossible  depth  of  85  ply  would  be  needed  to  find  the  best  defense  from 
some  positions.  With  the  additional  advice  not  to  lose  the  knight  and  to  keep  king  and  knight 
together,  search  to  about  10  ply  is  sufficient.  With  the  further  refinements  included  in  the 
actual  Advice  Table,  the  program  Is  reported  to  play  the  KNKR  endgame  at  master  level  using 
only  a 4-ply  search. 
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0.  Example  Search  Programs 
D1.  Logic  Theorist 

The  Logic  Theorist  (LT)  was  a program  written  by  Allen  Newell,  J.  C.  Shaw,  and  H.  A. 
Simon  In  1956,  as  a joint  project  of  the  RAND  Corporation  and  the  Carnegie  Institute  of 
Technology.  It  was  one  of  the  earliest  programs  to  investigate  the  use  of  heuristics  In 
problem  solving.  The  term  heuristics,  as  used  by  Newell,  Shaw  and  Simon,  referred  to  "the 
complex  processes  . . . that  are  effective  in  problem-solving.”  They  stated, 

We  are  not  interested  in  methods  that  guarantee  solutions,  but  which 
require  vast  amounts  of  computation.  Rather,  we  wish  to  understand 
how  a mathematician,  for  example,  is  able  to  prove  a theorem  even 
though  he  does  not  know  when  he  starts  how,  or  If,  he  Is  going  to 
succeed.  (Newell,  Shaw,  & Simon,  1963b,  p.  109) 

Heuristics  were  thus  identified  with  processes  “that  may  solve  a given  problem,  but  offer  no 
guarantee  of  doing  so"  (p.  114;  see  also  Overview). 

In  descriptions  of  the  Logic  Theorist  program,  the  heuristics  discussed  by  Newell,  Shaw, 
and  Simon  relate  principally  to  limiting  the  search  space  by  means  of  an  apt  problem 
formulation.  Within  the  defined  space,  the  search  was  blind  except  for  some  minor 
selectivity  In  the  selection  of  operators  (see  C3a). 

The  problem  domain  of  the  Logic  Theorist  Is  the  proof  of  theorems  in  the  propositional 
calculus  (see  Pepresentetion.Logic).  The  basis  is  Whitehead  and  Russell's  Principia 
Mathematics,  from  which  both  axioms  and  theorems  to  be  proved  were  taken.  There  are 
five  axioms,  as  follows; 

r 1.  (P  V P)  3 P 

2.  p 3 (q  V p) 

3.  (p  v q)  a (q  V p) 

4.  [p  v (q  v r)]  = [q  v (p  v r)] 

6.  (p  3 q)  3 [(r  v p)  3 (r  v q)] 

Some  typical  theorems  that  LT  was  given  to  prove  include: 

2.01.  (p  3 ~p)  a ~p 

2.46.  ~(p  v q)  a ~p 

2.31.  [p  v (q  v r)]  3 [(p  v q)  v r] 

The  numbering  of  the  theorems  is  taken  from  Whitehead  and  Russell.  In  some  cases,  the 
data  given  the  program  Included  not  only  the  axioms  but  also  previously  proved  theorems 
from  that  work.  When  all  earlier  theorems  were  included  with  the  axioms,  the  program 
succeeded  in  proving  38  of  the  first  52  theorems  in  Chapter  2 of  Principle  Mathematic  a,  In 
the  sequence  given  there. 

The  program  operates  by  reasoning  backward,  from  the  theorem  to  be  established,  to  the 
axioms  and  given  theorems.  Three  operators  were  provided  for  reducing  the  theorem  to  be 
proved,  let  us  say  X,  to  an  axiom  or  theorem.  These  operators  were: 
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Detachment:  To  show  X,  find  an  axiom  or  theorem  of  the  form  A a X,  and 
transform  the  problem  to  the  problem  of  showing  A. 

Forward  chaining:  To  show  X where  X has  the  form  A 3 C,  find  an  axiom  or 
theorem  of  the  form  A 3 B,  and  transform  the  problem  to  the  problem  of 
showing  B o C. 

Backward  chaining:  To  show  X where  X has  the  form  A o C,  find  an  axiom 
or  theorem  of  the  form  C,  and  transform  the  problem  to  the  problem  of 
showing  A 3 B. 

Since  the  axioms  and  given  theorems  contain  variables,  consideration  must  be  given  to 
the  means  for  deciding  whether  a problem  has  in  fact  been  reduced  to  something  known. 
The  question  Is  whether  a current  problem  expression  X is  an  instance  of  an  axiom  or  known 
theorem.  The  test,  called  the  Substitution  Test,  uses  two  rules  of  Inference  distinct  from 
those  reflected  in  the  operators: 


Substitution:  A variable  in  a theorem  may  be  replaced,  in  all  its 
occurrences  throughout  the  theorem,  by  an  expression.  For  example, 
substituting  the  expression  "p  v q"  for  the  variable  "p"  transforms 


into 


P = (q  v p) 

(p  v q)  a [q  v (p  v q)]. 


Replacement:  The  connective  "3"  is  interchangeable  with  its  definition. 
That  is,  if  p and  q are  expressions,  then 


can  be  replaced  by 
and  vice  versa. 


p 3 q 
~p  v q 


As  well  as  being  used  to  determine  whether  a proof  Is  complete,  the  substitution  test  is  also 
essential  for  determining  what  applications  of  the  three  operators  are  possible  with  respect 
to  a given  problem  expression. 

The  general  algorithm  used  by  the  Logic  Theorist  is  a blind,  breadth-first  state-space 
search  using  backward  reasoning.  The  initial  state  corresponds  to  the  original  theorem  to  be 
proved.  To  test  whether  an  expression  has  been  proved,  the  program  applies  the 
substitution  test,  pairing  the  problem  expression  with  each  axiom  and  assumed  theorem,  in 
turn.  If  substitution  falls,  the  expression  is  placed  on  a list  of  open  problems;  problems  are 
selected  from  this  list  to  become  the  current  problem  in  first-in,  first-out  order. 

To  a problem  selected  from  the  list,  each  of  the  three  operators  is  applied,  in  fixed 
order  and  In  all  possible  ways,  to  generate  new  open  problems.  The  search  terminates  with 
success  as  soon  as  a single  problem  is  generated  that  passes  the  substitution  test,  since 
this  means  that  a path  has  been  completed  between  an  axiom  and  the  original  problem.  The 
search  fails  if  It  exceeds  time  or  space  limits,  or  if  it  runs  out  of  open  problems. 

An  example  of  a case  in  which  the  latter  occurs  Is  the  attempted  proof  of  the  theorem 
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p or  *— —p  . 

To  succeed  with  this  proof,  LT  would  have  needed  more  powerful  operators;  this  particular 
problem  required  the  ability,  which  LT  lacked,  to  transform  a problem  to  a set  of  subproblems, 
or  conjunctive  subgoals,  which  all  had  to  be  solved  in  order  to  solve  the  original  problem. 

There  are  some  qualifications  to  the  preceding  general  description  of  LT.  One  concerns 
the  statement  that  each  operator  Is  applied  to  the  current  problem  In  every  possible  way, 
that  is,  that  the  current  problem  expression  Is  matched  against  every  axiom  and  assumed 
theorem  to  determine  the  applicability  of  any  of  the  operators  to  that  expression-axiom  pair. 
In  fact,  the  program  attempted  a match  for  the  purpose  of  discovering  an  appropriate 
substitution  only  if  the  pair  had  passed  a test  Indicating  equality  of  certain  gross  features, 
such  as  the  number  of  distinct  variables  in  each.  This  test  for  similarity  occasionally 
rejected  a pair  for  which  a substitution  in  fact  would  have  been  possible,  thus  excluding  a 
proof  the  program  would  otherwise  have  found.  Overall,  the  utility  of  this  similarity  test  was 
considered  rather  marginal. 

Some  other  additions,  apparently  made  In  a later  version  of  the  program  (see  Newell  & 
Simon,  1972,  pp.  125-128),  included  (a)  ordering  the  open  problems,  taking  up  those 
involving  simpler  expressions  first  instead  of  proceeding  in  a strictly  breadth-first  order,  and 
(b)  rejecting  some  subproblems  entirely  as  too  complicated  or  apparently  unprovable.  In  the 
implementation  of  these  features,  the  latter  appeared  to  be  the  more  effective  measure  in 
reducing  search  effort.  There  was  also  experimentation,  as  mentioned  previously,  with  the 
number  of  theorems  that  could  be  assumed  as  given  in  addition  to  the  basic  axioms.  The 
conclusion  on  this  point  was  that  "a  problem  solver  may  be  encumbered  by  too  much 
information.  Just  as  he  may  be  handicapped  by  too  little"  (Newell  & Simon,  1972,  p.  127). 


References 

See  Newell,  Shaw,  & Simon  (1963b),  Newell  & Simon  (1972),  and  Whitehead  & Russell 
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D2.  General  Problem  Solver 

The  General  Problem  Solver  (GPS)  was  developed  by  Allen  Newell,  J.  C.  Shaw,  and  H.  A. 
Simon  beginning  in  1957.  The  research  had  a dual  Intention:  It  was  aimed  both  at  getting 
machines  to  solve  problems  requiring  intelligence  and  at  developing  a theory  of  how  human 
beings  solve  such  problems.  GPS  was  the  successor  of  the  authors'  earlier  Logic  Theorist 
program  (Article  01),  whose  methods  had  only  a slight  resemblance  to  those  used  by  humans 
working  on  similar  problems.  Development  of  GPS  continued  through  at  least  ten  years  and 
numerous  versions  of  the  program.  The  final  version,  described  in  detail  in  Ernst  and  Newell 
(1969),  was  concerned  with  extending  the  generality  of  the  program,  not  with  the 
psychological  theory. 

The  name  "General  Problem  Solver"  came  from  the  fact  that  GPS  was  the  first  problem- 
solving program  to  separate  its  general  problem-solving  methods  from  knowledge  specific  to 
the  type  of  task  at  hand.  That  is.  the  problem-solving  part  of  the  system  gave  no  information 
about  the  kind  of  task  being  worked  on;  task-dependent  knowledge  was  collected  in  data 
structures  forming  a task  environment.  Among  the  data  structures  were  objects  and  operators  for 
transforming  objects.  A task  was  normally  given  to  GPS  as  an  initial  object  and  a desired 
object,  Into  which  the  Initial  object  was  to  be  transformed.  GPS  objects  and  operators  were 
similar  to  the  states  and  operators  of  a state-space  problem  representation  (Article  B1). 

The  general  problem-solving  technique  introduced  by  GPS,  however,  does  not  fit  neatly 
Into  either  the  state-space  or  the  problem-reduction  representation  formalisms.  It  dif f : ^ 
from  a standard  state-space  search  (e.g.,  Article  1 1C  1 ) in  the  way  it  decides  what  path  to 
next.  This  technique,  called  means-ends  analysis , is  a major  theoretical  contribution  of  the 
program.  It  assumes  that  the  differences  between  a current  object  and  a desired  object  can 
be  defined  and  classified  into  types  and  that  the  operators  can  be  classified  according  to 
the  kinds  of  difference  they  might  reduce.  At  each  stage,  GPS  selects  a single  relevant 
operator  to  try  to  apply  to  the  current  object.  The  search  for  a successful  operator 
sequence  proceeds  depth  first  as  long  as  the  chosen  operators  are  applicable  and  the  path 
shows  promise.  Backup  Is  possible  If  the  current  path  becomes  unpromising — for  example,  if 
eliminating  one  difference  has  introduced  a new  one  that  is  harder  to  get  rid  of. 

An  Important  feature  of  means-ends  analysis  is  the  fact  that  the  operator  selected  as 
relevant  to  reducing  a difference  may  in  fact  be  inapplicable  to  the  current  object.  Rather 
than  rejecting  the  operator  for  this  reason,  GPS  attempts  to  change  the  current  object  Into 
an  object  appropriate  as  Input  to  the  chosen  operator.  The  result  of  this  strategy  is  a 
recursive,  goal-directed  program  that  records  the  search  history  In  an  AND/OR  graph  (Article 
B2)  with  partial  development  of  nodes  (Article  C3a). 


Goals  and  Methods 

The  most  important  data  structure  used  by  GPS  Is  the  goal.  The  goal  Is  an  encoding  of 
the  current  situation  (an  object  or  list  of  objects),  the  desired  situation,  and  a history  of  the 
attempts  so  far  to  change  the  current  situation  into  the  desired  one.  Three  main  types  of 
goals  are  provided: 

1 . Transform  object  A into  object  B. 

2.  Reduce  a difference  between  object  A and  object  B by  modifying  object  A. 

3.  Apply  operator  Q to  object  A. 
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Associated  with  the  goal  types  are  methods,  or  procedures,  for  achieving  them.  These 
methods,  shown  in  a simplified  version  In  Figure  1,  can  be  understood  as  problem-reduction 
operators  that  give  rise  either  to  AND  nodes,  in  the  case  of  transform  or  apply,  or  to  OR 
nodes  in  the  case  of  a reduce  goal.  The  initial  task  presented  to  GPS  is  represented  as  a 
transform  goal,  in  which  A is  the  Initial  object  and  B the  desired  object. 


TRANSFORM 
A TO  B 


REDUCE  DIFFERENCE 
BETWEEN  A AND  B, 


^TRANSFORM 
A*  TO  B 


GIVING  OUTPUT  A' 


REDUCE  DIFFERENCE 
BETWEEN  A AND  B 


SELECT  A RELEVANT  OPERATOR  Q 
AND  APPLY  IT  TO  A 
GIVING  OUTPUT  A' 


APPLY  OPERATOR  Q TO  A 


REOUCE  DIFFERENCE 
BETWEEN  A AND  THE 
PRECONDITIONS  FOR 
Q,  GIVING  OUTPUT  A" 


APPLY  0 TO  A". 
GIVING  OUTPUT  A' 


Figure  1.  The  three  GPS  methods  for  problem  reduction. 

The  recursion  stops  If  the  goal  Is  primitive--that  is,  if  for  a transform  goal  there  is  no 
difference  between  A and  B;  and  if  for  an  apply  goal  the  operator  Q is  immediately 
applicable.  For  a reduce  goal,  the  recursion  may  stop,  with  failure,  when  all  relevant 
operators  have  been  tried  and  have  failed. 


Selection  of  Operators 

In  trying  to  transform  object  A to  object  B,  the  transform  method  uses  a matching 
process  to  discover  the  differences  botween  the  two  objects.  The  possible  types  of 
difference  are  predefined  and  ordered  by  estimated  difficulty,  for  each  kind  of  task.  The 
most  difficult  difference  found  Is  the  one  chosen  for  reduction.  A domain-dependent  data 
structure  called  the  Table  of  Connections  lists  the  operators  relevant  to  reducing  each 
difference  type. 


Depth  Bounds 

Several  heuristics  are  provided  to  prevent  GPS  from  following  a false  path  indefinitely. 
Some  of  the  bases  on  which  a current  goal  may  be  abandoned,  at  least  temporarily,  are  the 
following: 

1.  Each  goal  should  be  easier  than  Its  parent  goal. 


74 


Al  Handbook 


2.  Of  a pair  of  AND  nodes  representing  subgoals  generated  by  transform  or 
apply,  the  second  subgoal  attempted  should  bo  oaslor  than  tho  first. 

3.  A newly  generated  object  should  not  bo  much  largor  than  tho  objocts 
occurring  in  tho  topmost  goal. 

A.  Once  a goal  has  boon  generated,  tho  identical  goal  should  not  bo 
generated  again. 


An  Example 

lho  first  task  environment  to  which  GPS  was  applied  was  tho  domain  of  the  Logic 
Theorist:  proving  theorems  In  tho  prepositional  calculus.  The  initial  and  dosirod  objocts  were 
expressions,  one  to  bo  transformed  Into  tho  other  by  moans  of  operators  representing  rules 
of  Inference.  Thoro  wore  twelve  operators  altogether,  Including  tho  following  rules.  (Tho 
symbol  means  "may  be  rewritten  as.") 


Rule  1 . 

Avll 

*«>  b v A 

a a a 

==>  B a A 

Mule  6. 

A v b 

<“>  ~(~A  A ~b) 

Rule  6. 

A 3 b 

<**>  ~AvB 

Six  possible  difference  typos  wore  recognized: 

(a)  occurrence  of  a variable  in  one  expression  but  not  tho  other, 

(b)  occurrence  of  a variable  a different  number  of  times  In  the  two 
expressions, 

(c)  difference  In  sign, 

(d)  dlfforonco  In  binary  connective, 

(o)  difference  In  grouping,  and 

(f)  dlfforonco  In  position  of  components. 

The  list  Just  gl>  en  Is  In  decreasing  order  of  assumod  difficulty.  Evory  difference  between 
main  expressions,  howovor,  was  considered  moro  difficult  than  any  difference  between 
subexpressions. 

»V,»h  this  background,  a tiaco  (slightly  simplified)  of  GPS's  performance  on  a simple 
~ • «'<  t>iw  can  tm  given  The  problem  Is  to  transform  tho  Initial  expression 

R A (~P  D 0)  . 


u,i  ' ' in,,  desired  expression 


(QvP)aR, 


shown  below. 


Search 


76 


Goal  1:  Transform  LI  into  LO. 

Goal  2:  Reduce  positional  difference  botwoon  LI  anil  LO. 
Goal  3:  Apply  Rule  1 to  LI. 

Return  L2:  ( ~P  a G)  a R 

Goal  4:  Transform  L2  into  10. 

Goal  6:  Reduce  difference  In  connective  botwoon 
loft  subexpressions  of  L2  and  LO. 

Goal  6:  Apply  Rule  6 to  loft  part  of  L2. 

Goal  7.  Reduce  difference  in  connective 
botwoon  loft  part  of  L2  and 
precondition  for  Rule  6. 

Reject  goal  7 as  no  easier  than  goal  6. 

Goal  3:  Apply  Rule  6 to  loft  part  of  L2. 

Return  l 3;  (P  v G)  a R 

Goal  0:  Transform  L3  Into  LO. 

Goal  10:  Reduce  positional  difference 
botwoon  loft  parts  of  Lit  and  LO. 

Goal  1 1:  Apply  Rule  1 to  loft  part  of  L3. 

Return  14:  (Q  v P)  a R 

Goal  12:  Transform  L4  to  LO. 

No  difference  exists,  so  problem  is  solved. 


The  Problem  of  Generality 

GPS  was  Intended  to  model  generality  in  problem  solving  through  use  of  the  broadly 
applicable  techniques  of  heuristic  search,  and  the  stiategy  of  means-ends  analysis  In 
particular.  The  Implementation  of  these  techniques  was  dependent  on  the  Internal 
representation  of  objects  and  operators  These  representations,  In  early  versions  of  GPS. 
wore  nicely  suited  to  logic  tasks  like  the  example  above.  Hut  they  wore  Inadequate  for 
many  other  kinds  of  heuristic  search  problems,  before  trnst's  extensions  to  the  piogmin 
(Frnst  A Newell,  1060),  GPS  had  in  tact  solved  only  two  problems  outside  the  logic  domain 

The  objoct  of  l rust's  work  was  to  extend  the  number  of  kinds  of  problems  that  GPS 
could  handle  while  holding  Its  power  at  a constant  level  One  of  his  generalizations  was  In 
the  representation  of  objects.  I nrlior,  a desired  object  had  had  to  be  specified  by  giving  Its 
exact  form.  Forms  containing  variables  and  lists  of  forms  could  be  used  if  noeessai y Rut 
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these  too  were  Inadequate  for  representing  symbolic  integration  problems,  in  which  the 
desired  object  is  any  form  whatever  that  does  not  contain  an  integral  sign.  Hence  the 
description  of  a desired  object  by  a list  of  constraints  was  introduced. 

Another  change  was  in  the  representation  of  operators,  originally  specified  by  giving 
the  form  of  the  Input  object  and  the  form  of  the  resulting  output  object.  For  some  kinds  of 
problems,  It  was  desirable  to  hove  other  tests  of  applicability  besides  the  form  of  the  input 
object,  and  to  be  able  to  describe  the  output  object  as  a function  of  the  input.  A third 
change  enabled  GPS  to  deal  with  unordered  sots  of  symbols,  eliminating  the  need  for  special 
operators  to  permute  their  elements. 

The  generalized  program  succeeded  in  solving  problems  of  1 1 different  kinds  including 
symbolic  integration,  resolution  theorem  proving,  and  a variety  of  puzzles.  Each 
generalization,  however,  entailed  changes  in  the  ways  the  problem  representations  could  bo 
processed,  and  these  led  In  turn  to  deterioriation  with  respect  to  the  kinds  of  differences 
that  could  bo  detected.  The  only  representable  differences  became  "local"  ones.  An 
example  of  a global  difference,  which  GPS  could  no  longer  recognize,  was  the  total  number  of 
times  a variable  occurred  in  a logic  formula.  Consequently,  theorem  proving  in  the 
propositional  calculus  was  not  among  the  eleven  tasks  that  the  final  version  of  GPS  could  do. 

In  the  task  domains  in  which  GPS  did  succeed,  it  could  solve  only  simple  problems;  and 
those,  less  efficiently  than  special-purpose  problem  solvers.  If  a long  search  was  required, 
It  ran  out  of  memory  space;  and  even  easy  problems,  if  they  needed  objects  as  complex  as  a 
chess  position,  quickly  exhausted  memory  on  a machine  with  65K  words.  But  GPS  was  not 
expocted  to  be  a performance  program.  What  it  yielded,  in  its  authors'  view,  was  "a  series 
of  lessons  that  give  a more  porfect  view  of  the  nature  of  problem  solving  and  what  Is 
required  to  construct  processes  that  accomplish  it"  (Ernst  & Newell,  1969,  p.  2).  Although 
additional  generalizations,  such  as  game  playing,  were  considered  feasible,  the  authors 
concluded  that  GPS  needed  no  further  programming  accretions  and  recommended  that  it  be 
laid  to  rest. 
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See  Ernst  & Newell  (1969),  Nowell  & Ernst  (1965),  Newell,  Shaw,  & Simon  (1960), 
Newell  & Simon  (1963),  and  Newell  & Simon  (1972). 


03.  Gelernter's  Geometry  Theorem-proving  Machine 

Horbert  Gelerntor's  geometry  tlioorem-proving  machine  was  a program  written  In  1959 
at  the  IBM  Research  Center  In  New  York.  The  program  was  written  In  an  extended  FORTRAN, 
the  FORTRAN  List  Processing  Language,  and  implemented  on  an  IBM  704  computer.  The 
purpose  of  the  program  was  to  solvo  problems  taken  from  high-school  textbooks  and  final 
examinations  In  plane  goomotry.  As  with  Nowoll,  Shaw,  and  Simon's  Logic  Theorist,  which 
proved  thoorems  In  the  propositional  calculus,  the  fact  that  there  were  algorithms  for  solving 
problems  in  these  domains  was  considered  irrelevant,  since  the  object  was  to  explore  the 
use  of  heuristic  methods  in  problem  solving. 

The  formal  system  within  which  the  goomotry  program  worked  contained  axioms  on 
parallel  lines,  congruence,  and  equality  of  segments  and  angles.  This  set  of  axioms,  which 
was  not  meant  to  be  either  complete  or  nonredundant,  was  along  the  lines  of  an  elementary 
textbook.  The  axioms  ployed  the  rolo  of  problem-reduction  operators.  Some  examples  are: 
(a)  To  show  that  two  line  segments  are  equal,  show  that  they  are  corresponding  elements  of 
congruent  triangles;  (b)  to  show  that  two  angles  ore  equal,  show  that  they  are  both  right 
angles;  and  (c)  to  show  that  two  triangles  are  congruent,  show  the  equality  of  a side  and 
two  angles  in  corresponding  positions,  or  of  on  angle  and  two  sides.  The  operators  for 
establishing  congruence  split  tho  problem  into  three  subprobloms,  each  to  bo  solved 
separately  by  showing  equality  for  ono  pair  of  olomonts.  Nowell  and  Simon  (1972,  p.  138) 
Indicate  that  the  geometry  machine  was  the  first  program  that  was  able  to  handle  conjunctive 
subgoals.  The  program  works  backwards  from  the  theorem  to  bo  proved,  recording  Its 
progress  In  what  amounted  to  an  AND/OR  tree  (Article  B2). 

Some  examples  of  problems  solved  by  tho  program  wore  the  following: 

1.  Given  that  angle  ABD  equals  angle  DBC,  that  sogment  AD  is  perpendicular  to  segment 
AB,  and  that  segment  DC  is  perpendicular  to  segment  BC,  show  that  AD  equals  CD. 


Figure  1.  Diagram  for  problem  1. 


2.  Given  that  ABCD  is  a quadrilateral,  with  sogment  BC  parallel  to  segment  AD  and  with 
BC  equal  to  AD,  show  that  segment  AB  equals  segment  CD. 


Figure  2.  Diagram  for  problem  2. 
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A problom  was  given  to  the  program  in  the  form  of  a statement  describing  the  premises 
and  the  goal.  A proof  was  a sequence  of  statements  giving  the  reduction  of  the  goal  to 
trivial  goals--ordinarily,  goals  to  establish  an  already  established  formula.  One  feature  used 
to  reduce  the  search  effort  needed  to  find  a proof  was  the  recognition  of  syntactic  symmetry. 
Some  examples  of  symmetric  pairs  of  goals  are  the  following: 

a.  If  d(x,y)  is  the  distance  from  point  x to  point  y,  then  d(A,B)  = d(C,D)  Is 
symmetric  with  d(D,C)  = d(A,B). 

b.  If  ABCO  is  a parallelogram  and  point  E is  the  intersection  of  its  diagonals  AC 
and  BD,  then  d(A,E)  = d(E,C)  is  symmetric  with  d(S,E)  = d(E,D). 

The  recognition  of  symmetry  was  used  in  two  ways.  First,  if  a given  goal  was  ever  reduced 
to  a subgoal  symmetric  with  it,  the  subgoal  could  be  rejected  as  representing  circular 
reasoning.  Second,  if  parallel  goals  A and  B were  syntactically  symmetric  and  goal  A had 
been  established,  then  goal  B could  be  established  by  symmetry— in  effect  by  saying,  for  the 
second  half  of  the  proof,  "Similarly,  B." 

The  most  notable  feature  of  the  program,  however,  was  an  additional  part  of  the 
problem  statement  used  to  avoid  attempting  proofs  by  blind  syntactic  manipulation  alone. 
This  input  was  a diagram,  similar  to  Figures  1 and  2 (although  specified  by  lists  of 
coordinates),  of  the  points  and  line  segments  mentioned  in  the  theorem.  The  particular  input 
figure  was  chosen  to  avoid  spurious  coincidences  and  reflect  the  greatest  possible 
generality.  Whenever  a subgoal  was  generated,  it  was  checked  for  consistency  with  the 
diagram.  If  false  in  the  diagram,  the  subgoal  could  not  possibly  be  a theorem  and  therefore 
could  be  pruned  from  the  search  tree.  A slight  qualification  is  that  finite  precision  arithmetic, 
applied  to  the  diagram,  occasionally  caused  a provable  subgoal  to  be  pruned  erroneously;  but 
It  was  reported  that  the  program  had  found  other  paths  to  the  solution  in  such  cases.  It  was 
estimated  that  the  use  of  a diagram,  together  with  the  discard  of  subgoals  representing 
circular  reasoning,  eliminated  about  095  out  of  every  thousand  subgoals. 

The  diagram  also  served  a second  purpose:  It  provided  an  additional  criterion  by  which 
a problem  could  be  considered  primitive.  For  example,  a rigorous  proof  of  the  theorem  in 
problem  1 would  require  showing  that  DB  is  a line  segment  and  that  BCD  and  BAD  are 
triangles.  The  axioms  needed  would  have  been  (a)  if  X and  Y are  distinct  points,  then  XY  is  a 
line  sogment;  and  (b)  If  X,  Y,  and  Z are  throe  distinct  non-collinear  points,  then  XYZ  is  a 
triangle.  For  a limited  class  of  such  properties,  the  program  did  not  require  formal  proof  but 
rather  considered  them  established  if  they  were  true  in  the  diagram.  It  recorded  explicitly 
the  assumptions  that  lied  been  made  based  on  the  diagram. 

The  central  loop  of  the  program  repeatedly  selects  the  next  goal  to  try.  Two  heuristics 
were  Included  for  goal  selection.  One  gave  highest  priority  to  classes  of  goals,  such  as 
Identities,  that  could  usually  be  established  in  one  step.  The  second  assigned  a "distance" 
between  the  goal  statement  and  the  set  of  premise  statements;  after  the  one-step  goals 
had  been  developed,  the  remaining  goals  were  selected  in  order  of  increasing  distance  from 
the  premise  set. 

Once  a goal  was  chosen  for  development,  the  action  taken  depended  on  its  status. 
Ordinarily,  it  would  be  reduced  to  subgoals  and  the  subgoals,  if  consistent  with  the  diagram 
but  not  sufficient  to  establish  the  current  goal  immediately,  would  be  added  to  the  list  of 
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goals  to  try.  If  no  new  acceptable  subgoals  were  generated,  the  program  checked  whether 
a construction  was  possible— a construction  being  the  addition  to  the  premises  of  the  problem 
of  a line  segment  between  two  existing  but  previously  unconnected  points.  The  new 
segment  would  be  extended  to  its  intersections  with  other  segments  in  the  figure.  New 
points  could  be  added  to  the  premises  only  if  generated  by  such  Intersections. 

A goal  for  which  a construction  was  found  possible  was  saved— to  be  tried  again  later 
if  all  goals  not  requiring  construction  should  be  exhausted.  If  the  goal  was  later  selected  for 
a second  try,  a construction  would  bo  made  and  the  problem  started  over  with  an  expanded 
premise  set.  An  example  of  the  use  of  this  technique  occurs  In  problem  2,  where  In 
considering  the  goal  AB  ■ CD,  the  program  generated  a subgoal  of  showing  that  triangles  ABD 
and  CDB  were  congruent.  The  subgoal  makes  sense  only  if  a line  segment  BD  exists,  so  the 
segment  Is  constructed,  and  the  proof  eventually  succeeds. 
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04.  Symbolic  Integration  Programs 


Slagle's  SAINT 

James  Slagle's  SAINT  program  (Symbolic  Automatic  INTegrator)  was  written  as  a 1961 
doctoral  dissertation  at  MIT.  The  program  solves  elementary  symbolic  Integration  problems— 
mainly  Indefinite  integration— at  about  the  level  of  a good  college  freshman.  SAINT  was 
written  In  LISP  and  run  interpretively  on  the  IBM  7090. 

The  kinds  of  questions  Slagle  intended  his  thesis  to  address  were  some  of  the  earliest 
questions  for  Al.  They  included,  for  example,  "Can  a computer  recognize  the  kinds  of 
patterns  that  occur  in  symbolic  expressions?  Just  how  important  is  pattern  recognition?  . . . 
Can  Intelligent  problem  solving  behavior  really  be  manifested  by  a machine?"  (Slagle,  1961, 
p.  9).  The  domain  of  symbolic  integration  was  chosen  as  a source  of  well-defined,  familiar, 
but  nontrivial  problems  requiring  the  manipulation  of  symbolic  rather  than  numerical 
expressions. 

The  Integration  problems  that  SAINT  could  handle  could  have  only  elementary  functions 
as  Integrands.  These  functions  were  defined  recursively  to  comprise  the  following: 

1.  Constant  functions. 

2.  The  Identity  function. 

3.  Sum,  product,  and  power  of  elementary  functions. 

4.  Trigonometric,  logarithmic,  and  inverse  trigonometric  functions  of 
elementary  functions. 

Throe  kinds  of  operations  on  an  Integrand  were  available: 

1.  Recognize  the  integrand  as  on  instance  of  a standard  form,  thus  obtaining  the 
result  Immediately  by  substitution.  Twenty-six  such  standard  forms  were 
used.  A typical  one  indicated  that  if  the  integrand  has  the  form  cv  dv,  the 
form  of  the  solution  is  (cv)/(ln  c). 

2.  Apply  an  "algorithm-like  transformation"  to  the  integral—that  is,  a 
transformation  that  Is  almost  guaranteed  to  be  helpful  whenever  it  can  be 
applied.  Eight  such  transformations  were  provided,  Including  (a)  factoring  out 
a constant  and  (b)  decomposing  the  Integral  of  a sum  Into  a sum  of  integrals. 

3.  Apply  a "heuristic  transformation"— that  Is,  a transformation  carrying 
significant  risk  such  that,  although  applicable,  it  might  not  be  the  best  next 
step.  The  10  heuristic  transformations  included  certain  substitutions  and  the 
technique  of  Integration  by  parts.  One  technique  that  was  not  implemented 
was  the  method  of  partial  fractions. 

The  program  starts  with  the  original  problem  as  a goal,  specified  as  an  Integrand  and  a 
variable  of  Integration.  For  any  particular  goal,  the  strategy  is  first  to  try  for  an  Immediate 
solution  by  substitution  Into  a standard  form;  falling  that,  to  transform  It  by  any  applicable 
algorithm-like  transformation;  and  finally  to  apply  each  applicable  heuristic  transformation  in 
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turn.  Both  the  algorithm-like  and  the  heuristic  transformations,  however,  generate  new  goals, 
to  which  the  same  strategy  may  be  applied.  The  result  is  an  AND/OR  graph  of  goals  (Article 

BS). 

The  order  In  which  goals  are  pursued  by  SAINT  depends  heavily  on  what  operations  can 
be  applied  to  them.  At  the  level  of  heuristic  transformations,  the  algorithm  is  an  ordered 
search:  A list,  called  the  Heuristic  Goal  List,  keeps  track  of  goals  on  which  progress  can  be 
made  only  by  applying  heuristic  transformations--that  is,  integrands  that  are  not  of  standard 
form  nor  amenable  to  any  algorithm-like  transformation.  To  each  goal  on  this  list  is  attached 
an  estimate  of  the  difficulty  of  achieving  it.  The  measure  of  difficulty  used  is  the  maximum 
level  of  function  composition  In  the  integrand.  Other  characteristics  of  the  goal,  such  as 
whether  It  Is  a rational  function,  an  algebraic  function,  a rational  function  of  sines  and 
cosines,  and  the  like,  are  also  stored  as  an  aid  to  determining  which  heuristic  transformations 
will  in  fact  apply.  The  outer  loop  of  the  program  repeatedly  selects  the  goal  that  looks  the 
easiest  from  the  Heuristic  Goal  List,  expands  it  by  applying  all  applicable  heuristic 
transformations,  and  possibly,  as  a result  of  the  expansion,  adds  new  elements  to  the 
Heuristic  Goal  List.  The  program  terminates  with  failure  if  it  runs  out  of  heuristic  goals  to  work 
on  or  If  it  exceeds  a pre-set  amount  of  working  space. 

An  important  qualification  to  this  process  concerns  the  use  of  standard  forms  and 
algorithm-like  transformations.  As  soon  as  any  new  goal  is  generated  (or  the  original  goal 
read  In),  an  immediate  solution  of  it  is  attempted.  Tho  attempt  consists  of,  first,  checking 
whether  the  integrand  is  a standard  form;  if  It  is  not,  checking  whether  an  algorithm-like 
transformation  applies;  and  If  one  does,  applying  it  and  calling  the  immediate  solution 
procedure  recursively  on  each  goal  resulting  from  that  transformation.  When  the  recursion 
terminates,  either  the  generated  goal  has  been  achieved  or  there  is  a set  of  goals--the 
generated  goal  itself  or  some  of  its  subgoals— to  be  added  to  the  Heuristic  Goal  List.  During 
expansion  of  a node  (one  Iteration  of  the  outer  loop),  new  heuristic  goals  are  accumulated  in 
a temporary  goal  list;  only  after  expansion  is  complete  are  their  characteristics  computed 
and  the  additions  made  to  the  Heuristic  Goal  List. 

Whenever  a goal  Is  achieved,  tho  implications  of  its  achievement  are  immediately 
checked.  If  it  is  the  original  goal,  the  program  terminates  successfully.  Otherwise,  if  it  was 
achieved  by  substitution  into  a standard  form,  it  may  cause  the  achievement  of  one  or  more 
parent  goals  as  well.  If  it  was  achieved  by  solution  of  a sufficient  number  of  its 
subproblcms,  it  may  not  only  cause  its  parent  or  parents  to  be  achieved  in  turn,  but  may  also 
make  others  of  its  subproblcms,  which  havo  not  yet  been  solved,  superfluous.  These  checks 
are  Implemented  in  a recursive  process,  referred  to  as  "pruning  the  goal  tree,"  that  is 
initiated  as  soon  as  any  goal  is  achieved.  Thus  a heuristic  goal  can  be  achieved  without 
having  been  fully  expanded. 


Moses's  SIN 

A second  important  symbolic  integration  program,  SIN  (Symbolic  INtegrator),  was  written 
by  Jool  Moses  in  1969,  also  as  a doctoral  dissertation  at  MIT.  Its  motivation  and  its  strategy 
as  an  Al  effort  wore  quite  different  from  those  of  SAINT.  Whereas  Slagle  had  compared  tho 
behavior  of  SAINT  to  that  of  freshman  calculus  students,  Moses  aimed  at  behavior 
comparable  to  expert  human  performance.  Ho  viewed  SAINT  as  emphasizing  generality  in 
that  It  examined  mechanisms,  liko  heuristic  search,  that  are  useful  in  many  diverse  problem 
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domains.  SIN,  In  contrast,  was  to  emphasize  expertise  in  a particular,  complex  domain.  To  do 
this,  It  concentrated  on  problem  analysis,  using  more  knowledge  about  integration,  than  SAINT 
had  employed,  to  minimize  the  need  for  search.  In  fact,  Moses  did  not  view  SIN  as  a heuristic 
search  program.  Hence,  the  program  will  be  describod  only  briefly  here;  and  a second  part  of 
Moses's  dissertation,  a differential  equation  solver  called  SOLDIER,  will  not  be  described. 

SIN  worked  in  three  stages,  each  stage  being  capable  of  solving  harder  problems  than 
the  stage  before.  Stage  1 corresponded  roughly  to  Slagle's  immediate  solution  procedure 
but  was  more  powerful.  It  used  a table  of  standard  forms;  two  of  Slagle's  algorithm-like 
transformations;  and,  most  Importantly,  a method  similar  to  one  of  Slagle's  heuristic 
transformations,  referred  to  as  the  Derivative-divides  method.  The  idea  behind  this  grouping 
of  methods  was  that  they  alone  would  be  sufficient  to  solve  the  most  commonly  occurring 
problems,  without  invoking  the  computationally  more  expensive  machinery  of  the  later  stages. 

A problem  that  stage  1 could  not  solve  was  passed  on  to  stage  2.  This  stage 
consisted  of  a central  routine,  called  FORM,  and  1 1 highly  specific  methods  of  integration. 
(One  of  these  methods  was  a program  for  integrating  rational  functions  that  had  been  written 
by  Manove,  Bloom,  and  Engclman,  of  the  MITRE  Corporation,  in  1964.)  In  general,  the  task  of 
FORM  was  to  form  a hypothesis,  usually  based  on  local  clues  in  the  integrand,  about  which 
method,  if  any,  was  applicable  to  the  problem.  Only  rarely  did  more  than  one  method  apply. 
The  routine  chosen  first  tried  to  verify  its  applicability;  if  it  could  not,  it  returned  tc  let  FORM 
try  again.  If  the  routine  did  verify  the  hypothesis,  however,  SIN  then  became  committed  to 
solving  the  problem  by  that  method  or  not  at  all.  The  method  chosen  either  solved  the 
problem  using  mechanisms  internal  to  it  or  transformed  the  problem  and  called  SIN  recursively 
to  solve  the  transformed  problem. 

Stage  3 of  SIN  was  Invoked,  as  a last  resort,  only  if  no  stage  2 method  was  applicable. 
Two  general  methods  were  programmed  here.  One  method  was  integration-by-parts,  which 
used  blind  search,  subject  to  certain  constraints,  to  find  the  appropriate  way  to  factor  the 
integrand.  The  other  was  a nontraditional  method  based  on  the  Liouville  theory  of  integration 
and  called  the  EDGE  (EDucated  GuEss)  heuristic.  This  method  involved  guessing  the  form  of 
the  integral.  The  EDGE  heuristic  was  characterized  as  using  a technique  similar  to  means- 
ends  analysis,  if  its  guess  did  not  lead  directly  to  a solution. 


Performance  of  SAINT  and  SIN 

SAINT  was  tested  on  a total  of  86  problems,  54  of  them  chosen  from  MIT  final 
examinations  In  freshman  calculus.  It  succeeded  In  solving  all  but  two.  The  most  difficult 
problem  It  solved,  both  In  terms  of  time  and  the  number  of  heuristic  transformations  occurring 
In  the  solution  tree  (four),  was  the  integral  of 


(sec  t)2 


dt  . 


1 + (sec  t)2  - 3(tan  t) 

Slagle  proposed  additional  transformations  that  would  have  handled  the  two  failures,  which 

were  the  Integrals  of 


x(1*x)1/2dx  and  cos(x1/2)  dx  . 
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SIN,  in  contrast,  was  intended  to  model  the  behavior  of  an  expert  human  integrator. 
The  results  of  running  SIN  on  all  of  Slagle's  test  problems  were  that  more  than  half  were 
solved  In  stage  1,  and  all  but  two  of  the  rest  (both  of  which  used  integration  by  parts)  were 
solved  In  stage  2.  After  adjusting  for  the  facts  that  SAINT  and  SIN  ran  on  different  machines 
and  that  one  was  interpreted  and  the  other  compiled,  and  for  other  factors  making  the 
programs  difficult  to  compare,  Moses  estimated  that  SIN  would  run  on  the  average  about 
three  times  faster  than  SAINT.  Taking  into  account  a test  on  more  difficult  problems  as  well, 
he  expressed  the  opinion  that  SIN  was  "capable  of  solving  integration  problems  as  difficult 
as  ones  found  In  the  largest  tables"  (p.  140)  and  that  it  was  fast  and  powerful  enough  for 
use  in  "a  practical  on-line  algebraic  manipulation  system"  (p.  6).  For  later  developments  in 
this  direction,  see  Application6.Mac6yma. 
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06.  STRIPS 

STRIPS  Is  a problem-solving  program  written  by  Richard  Fikes  and  Nils  Nilsson  (1971 ) at 
SRI  International.  Each  problem  for  STRIPS  is  a goal  to  be  achieved  by  a robot  operating  in  a 
simple  world  of  rooms,  doors,  and  boxes.  The  solution  is  a sequence  of  operators,  called  a 
plan,  for  achieving  the  goal.  (For  a review  of  the  various  senses  of  the  word  plan,  see 
Planning).  The  robot's  actual  execution  of  the  plan  is  carried  out  by  a separate  program, 
distinct  from  STRIPS.  A later  (1972)  addition  to  the  basic  STRIPS  system  permits  plans  to  be 
generalized  and  used  again,  giving  the  system  some  capacity  for  learning. 


The  Basic  STRIPS  System 

The  world  model.  The  world  In  which  the  STRIPS  robot  works  consists  of  several 
rooms  connected  by  doors,  along  with  some  boxes  and  other  objects  that  the  robot  can 
manipulate.  STRIPS  represents  this  world  by  a set  of  well-formed  formulas  in  the  first-order 
predicate  calculus  (see  RepresentatioaLogic).  Some  formulas  in  the  world  model  are  static 
facts,  such  as  which  objects  are  pushable  and  which  rooms  are  connected.  Other  facts, 
such  as  the  current  location  of  objects,  must  be  changed  to  reflect  the  actions  of  the  robot. 

Operators.  The  primitive  actions  available  to  the  robot  are  represented  by  operators. 
Typical  operators  include  going  somewhere  and  pushing  an  object  somewhere,  the  locations 
being  given  as  parameters.  Each  operator  has  preconditions  to  its  applicability;  to  push  a box, 
for  example,  the  robot  must  first  be  next  to  the  box.  The  application  of  an  operator  is 
realized  by  making  changes  in  the  world  model.  The  appropriate  changes  are  given  by  a 
delete  list  and  an  add  list,  specifying  the  formulas  to  be  removed  from  and  added  to  the  world 
model  as  a result  of  the  operation.  Thus,  each  operator  explicitly  describes  what  it  changes 
in  the  world  model. 

A typical  operator  Is  G0T08,  which  denotes  the  robot's  going  up  to  an  object  in  the 
same  room: 

GOTOB  (bx)  "go  to  object  bx" 

Preconditions:  TYPE(bx, OBJECT)  and 

THERE  EXISTS  (rx)  [INROOM(bx.rx)  and  INROOM(ROBOT.rx)] 

Delete  list:  AT(ROBOT.V),  NEXTT0(R0B0T,“) 

Add  list:  NEXTTO(ROBOT.bx)  . 

The  precondition  statement  requires  that  bx  be  an  object  and  that  both  bx  and  the  robot  be 
in  the  same  room,  rx.  The  asterisks  in  the  delete  list  represent  arguments  with  any  values 

whatever. 

Method  of  operation.  STRIPS  operates  by  searching  a space  of  world  models  to  find 
one  In  which  the  given  goal  is  achieved.  It  uses  a state-space  representation  in  which  each 
state  Is  a pair  (world  model,  list  of  goals  to  be  achiovcd).  The  initial  state  is  (MO,  (GO)), 
where  MO  Is  the  Initial  world  model  and  GO  the  given  goal.  A terminal  state  gives  a world 
mo^ol  in  which  no  unsatisfied  goals  remain. 

Given  a goal  G (stated  as  a formula  in  the  predicate  calculus),  STRIPS  first  tries  to 
prove  that  G is  satisfied  by  the  current  world  model.  To  do  this,  the  program  uses  a modified 


version  of  the  resolution-based  theorem  prover  QA3  (Garvey  & Kling,  1969).  Typically  the 
proof  fails,  within  a pre-specified  resource  limit,  because  no  more  resolvents  can  be  formed 
(see  Theorem  Proving.Resolutian).  At  this  point,  STRIPS  needs  to  find  a different  world 
model  which  the  robot  can  achieve  and  which  satisfies  the  goal.  Because  this  task  is 
complicated  for  a simple  theorem  prover,  the  system  switches  to  a means-ends  analysis  similar 
to  that  of  GPS  (Article  02). 

To  do  the  means-ends  analysis,  the  program  extracts  a difference  between  the  goal  and 
the  current  model  and  selects  a “relevant"  operator  to  reduce  the  difference.  The 
difference  consists  of  any  formulas  from  the  goal  that  remain  outstanding  when  the  proof 
attempt  is  abandoned  (pruned,  if  this  set  is  large).  A relevant  operator  is  one  whose  odd  list 
contains  formulas  that  would  remove  some  part  of  the  difference,  thereby  allowing  the  proof 
to  continue. 

If  the  operator  is  applicable,  the  program  applies  it  and  tries  to  achieve  the  goal  in  the 
resulting  model;  otherwise,  the  operator's  precondition  becomes  a new  subgoal  to  be 
achieved.  Since  there  may  be  several  relevant  operators  at  each  step,  this  procedure 
generates  a tree  of  models  and  subgoals.  STRIPS  uses  a number  of  heuristics  to  control  the 
search  through  this  tree. 


An  Example  of  the  Basic  System's  Performance 

As  a simple  example,  suppose  the  robot  is  in  R00M1  and  the  goal  is  for  it  to  be  next  to 
BOX1,  which  Is  in  adjacent  R00M2.  The  initial  world  model  MO  contains  such  clauses  as 

INROOM  (ROBOT, R00M1), 

INROOM  (B0X1.R00M2), 

TYPE  (B0X1, OBJECT), 

CONNECTS  (D00R1 2.R00M1  ,R00M2), 

STATUS  (D00R1 2.0PEN),  . . , 

and  the  goal  GO  is 

G0  * NEXTTO  (ROBOT, B0X1)  . 

GO  is  not  satisfied,  and  the  difference  between  it  and  the  initial  model  is 
~NEXTTO  (ROBOT, B0X1 ).  STRIPS  determines  that  GOTOB  (bx),  defined  above,  is  a relevant 
operator,  with  bx  instantiated  as  B0X1.  The  operator  instance  GOTOB  (B0X1),  denoted  0P1, 
Is  not  immediately  applicable  (because  the  robot  is  in  the  wrong  room),  so  its  precondition  G1, 

G1  = TYPE  (B0X1, OBJECT)  and 

THERE  EXISTS  (rx)  [INROOM  (BOXl.rx)  and  INROOM  (ROBOT, rx)] 

becomes  a new  subgoal.  Relevant  operators  for  reducing  the  difference  between  G1  and 
the  initial  model  MO  are:  0P2  = GOTHRUDOOR  (dx,R00M2)  and 

0P3  = PUSHTHRUDOOR  (B0X1  ,dx,R00M1 ) (i.e„  move  the  robot  to  the  room  with  the  box,  or 
move  the  box  to  the  room  with  the  robot).  If  the  former  course  (the  better  one,  obviously)  is 
selected,  the  precondition 
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G2  = STATUS  (dx.OPEN)  and  NEXTTO  (ROBOT, dx)  and 

THERE  EXISTS  (rx)  [INROOM  (ROBOT, rx)  and  CONNECTS  (dx,rx,ROOM2)  ] 

is  the  new  subgoal.  The  difference  ~NEXTTO  (ROBOT.DOOR1 2)  can  be  reduced  by  the 
operator  OP4  = GOTODOOR  (DOOR12),  which  is  applicable  immediately.  Applying  0P4  adds  the 
clause  NEXTTO  (ROBOT, DOOR1 2)  to  the  model,  creating  a new  world  model  Ml.  G2  is  now 
satisfied  with  dx  = DOOR12,  so  OP2  can  be  instantiated  as  GOTHRUDOOR  (DOOR1 2.ROOM2) 
and  applied.  This  deletes  the  clause  INROOM  (ROBOT, ROOM  1 ) and  adds 
INROOM  (ROBOT, ROOM2).  G1  is  now  satisfied,  so  OP1  is  applied,  deleting 
NEXTTO  (ROBOT.DOOR12)  and  adding  NEXTTO  (ROBOT, BOX1 ),  the  desired  goal.  The  final  plan 
is  thus: 


OP 4:  GOTODOOR (DOOR 12) 

OP2:  GOTHRUDOOR  (DOOR1 2, ROOM2) 
OP1 : GCTOB  (BOX1 ) 


The  corresponding  solution  path  through  the  state  space  tree  is  as  follows: 

(MB,  ( G0 ) ) 

(M0,  (Gl,  GO)) 

(Mo)  (G2 , Gl,  GO ) ) 

\ 0P4 

(Ml,  (Gl,  G0) ) 

\ 0P2 
(M2.  (G0) ) 

\ 0P1 
(M3,  ()) 


Generalization  of  Plans 

In  the  basic  STRIPS  system,  each  new  problem  was  solved  from  scratch.  Even  if  the 
system  had  produced  a plan  for  solving  a similar  problem  previously,  it  was  not  able  to  make 
any  use  of  this  fact.  A later  version  of  STRIPS  provides  for  generalizing  plans  and  saving 
them,  to  assist  both  In  the  solution  of  subsequent  problems  and  also  In  the  intelligent 
monitoring  of  the  robot's  execution  of  the  particular  plan. 

Triangle  tables.  A specific  plan  to  be  generalized,  say  (OP1,  0P2 DPn),  is  first 

stored  in  a data  structure  called  a triangle  table.  This  is  a lower  triangular  array  representing 
the  preconditions  for  and  effects  of  each  operator  in  the  plan.  Some  of  its  properties  are 
the  following: 

1.  Cell  (1,0)  contains  clauses  from  the  original  model  that  are  still  true  when 
operator  I is  to  be  applied  and  that  are  preconditions  for  operator  I,  OPi. 
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2.  I1-' irked  (starred)  clauses  elsewhere  in  row  I are  preconditions  for  operator  i 
added  to  the  model  by  previous  operators. 

3.  The  effects  of  applying  operator  i are  shown  in  row  i+ 1 . The  operator's  add 
list  appears  in  cell  (i+1,  i).  For  each  previous  operator,  say  operator  j, 
clauses  added  by  operator  J and  not  yet  deleted  are  copied  Into  cell  (1+1, j). 

4.  The  add  list  for  a sequence  of  operators  1 through  i,  taken  as  a whole,  is 
given  by  the  clauses  in  row  i+1  (excluding  column  0). 

6.  T‘  c*  preconditions  for  a sequence  of  operators  i through  n,  taken  as  a whole, 
are  given  by  the  marked  clauses  in  the  rectangular  sub-array  containing  row  i 
and  cell  (n+1, 0).  This  rectangle  is  called  the  i-th  kernel  of  the  plan. 

The  triangle  table  for  the  previous  example  is  shown  below.  Operators  have  been 
renumbered  In  the  order  of  their  use. 


♦INROOM (ROBOT. 

ROOM!) 
♦C0NNECTS(D12. 
ROOM  1, ROOM 2) 

0P1 

G0T0D00R( D12 ) 

♦INR00M(R0B0T. 

R00M1 ) 
♦C0NNECTS(D12. 

ROOM  1 , R00M2  ) 
♦STATUS  (012. 
OPEN) 

♦NEXTTO 
(ROBOT. D1Z) 

0P2 

GOTHRUOOOR 

(D12.ROOM2) 

♦INR00M( B0X1 , 
R00M2 ) 
♦ TYPE(B0X1 , 

OBJECT) 

NEXTTO 
(ROBOT, D12) 

♦INR00M 
( ROBOT, R00M2) 

0P3 

GOTOB(BOXl) 

INR00M 

( ROBOT , R02M2  ) 

NEXTTO 
( ROBOT, BOX  1) 

0 


2 


3 


Figuro  1.  A triangle  table. 


Method  of  Generalization.  The  plan  is  generalized  by  replacing  all  constants  in  each  of 
the  clauses  In  column  0 by  distinct  parameters  and  the  rest  of  the  table  with  clauses  that 
assume  that  no  argument  to  on  operator  has  been  instantiated.  The  result  may  be  too 
general,  so  the  proof  of  the  preconditions  for  each  operator  Is  run  again,  noting  any 
substitutions  for  parameters  that  constrain  the  generality  of  the  plan.  Some  further 
corrections  are  made  for  remaining  overgeneralization,  which  might  make  the  plan  either 
inconsistent  or  inefficient  in  uso.  Finally,  the  generalized  plan,  termed  a MACROP,  is  stored 
away  for  future  use. 
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In  the  example  above,  the  generalized  plan  would  be 

GOTODOOR (dx) 

GOTHRUDOOR  (dx.rxl) 

GOTOB (bx) 

with  preconditions:  INROOM  (ROBOT, rx2) 

CONNECTS  (dx,rx2,rx1 ) 

STATUS  (dx.OPEN) 

INROOM  (bx.rxl) 

TYPE  (bx, OBJECT) 

and  add  list:  NEXTTO  (ROBOT, bx) 

INROOM  (ROBOT, rxl) 

That  is,  the  generalized  plan  sends  the  robot  from  any  room  through  a connecting  door  to  an 
object  in  the  adjacent  room. 

Using  the  MACROP  to  guide  execution.  When  STRIPS  produces  a detailed  plan  to 
achieve  a goal,  it  does  not  necessarily  follow  that  the  robot  should  execute  the  plan  exactly 
as  given.  One  possibility  Is  that  some  action  fails  to  achieve  its  expected  effect,  so  that 
the  corresponding  step  of  the  plan  needs  to  be  repeated.  Another  is  that  the  plan  found  is 
less  than  optimal  and  would  be  improved  by  omitting  some  steps  entirely.  The  necessary 
flexibility  during  execution  is  provided  by  using  the  MACROP  rather  than  the  detailed  plan  in 
monitoring  the  robot's  actions. 

At  the  beginning  of  execution,  the  parameters  of  the  MACROP  are  partially  instantiated 
to  the  case  at  hand.  The  robot  then  attempts,  at  each  stage,  to  execute  the  highest 
numbered  step  of  the  plan  whose  preconditions  are  satisfied.  This  procedure  omits 
unnecessary  steps  and  allows  repeated  execution,  possibly  with  changed  parameters,  of  a 
step  that  has  failed.  If  there  is  no  step  whose  preconditions  are  satisfied,  replanning 
occurs.  Determining  which  step  can  be  done  next  is  accomplished  by  a scan  that  exploits 
the  design  of  the  triangle  table. 

Using  MACROPs  in  planning.  When  STRIPS  is  given  a new  problem,  the  time  it  takes  to 
produce  an  answer  can  be  reduced  very  considerably  if  there  exists  a MACROP  that  can  be 
incorporated  Into  its  solution.  The  MACROP  given  above,  for  example,  could  be  used  as  the 
first  part  of  a plan  to  fetch  a box  from  an  adjacent  room.  The  part  of  the  MACROP  consisting 
of  its  first  two  suboperators,  if  used  alone,  would  also  give  a ready-made  solution  to  the 
problem  "Go  to  an  adjacent  room";  or  it  could  be  used  repeatedly  in  solving  "Go  to  a distant 
room." 

The  triangle  table  provides  the  means  of  determining  whether  a relevant  macro 
operator  exists.  To  determine  whether  the  sequence  of  operators  1 through  i of  the  MACROP 
is  relevant,  STRIPS  checks  the  add  list  of  this  sequence  as  given  by  the  i*1fA  row  of  the 
table.  Once  a MACROP  is  selected,  irrelevant  operators  ore  edited  out  by  a staightforward 
algorithm,  leaving  an  economical,  possibly  parameterized  operator  for  achieving  the  desired 
add  list.  The  operator's  preconditions  are  taken  from  the  appropriate  cells  of  column  0. 
Thus,  almost  any  sub-sequence  of  operators  from  a MACROP  can  become  a macro  operator  in 
a new  plan.  To  keep  new  MACROPs  from  producing  an  overwhelming  number  of  different 
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operators  that  must  be  considered  during  planning,  the  system  contains  provisions  f or 
provontlng  consideration  ot  redundant  parts  of  overlapping  MACROPs  and  for  deleting 
MACROPs  that  have  been  completely  subsumed  by  now  ones. 


In  a sequence  of  problems  given  to  STRIPS,  the  use  of  MACROPs  ir.  some  cases 
reduced  planning  timo  by  as  much  as  two-thirds.  The  longest  plan  so  formed,  consisting  of 
1 1 primitive  operations,  took  the  robot  from  one  room  to  a second  room,  opened  a door 
leading  to  a third  room,  took  tho  robot  through  tho  third  room  to  a fourth  room,  and  then 
pushed  two  pairs  of  boxes  togothor.  Ono  drawback  noted  by  the  authors  was  that,  howevor 
long  tho  solution  sequonco,  STRIPS  at  each  stago  of  its  search  dealt  with  ovory  operation  in 
complete  dotal).  A later  program,  Sacordoti's  ABSTRIPS  (Article  06),  provides  tho  mechanism 
for  deferring  tho  dotails  of  the  solution  until  after  its  main  outline  has  been  completed. 


References 

See  Fikos  & Nilsson  (1971),  Fikes,  Hart,  & Nilsson  (1072),  and  Garvey  & Kling  (1969). 
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06.  ABSTRIPS 

A combinatorial  explosion  faces  all  problem  solvers  that  attempt  to  use  heuristic  search 
In  a sufficiently  complex  problem  domain.  A technique  callod  hierarchical  search  or  hierarchical 
planning,  implemented  In  Earl  Sacerdoti's  ABSTRIPS  (1074),  is  on  attempt  to  reduce  the 
combinatorial  problem.  The  Idea  is  to  use  an  approach  to  problem  solving  that  can  recognize 
the  most  significant  features  of  a problem,  develop  an  outline  of  a solution  In  terms  of  those 
features,  and  deal  with  the  less  important  details  of  the  problem  only  after  the  outline  has 
proved  adequate. 

The  Implementation  of  this  approach  involvos  using  distinct  levels  of  problem 
representation.  A simplified  version  of  tho  problem,  from  which  details  have  been  omitted, 
occurs  In  a higher  level  problem  space  or  abstraction  space ; tho  detailed  version,  in  a ground  space. 
By  a slight  extension,  providing  for  sovcral  levels  of  detail  instead  of  just  two,  a hierarchy  of 
problem  spaces  is  obtained.  In  gonoral,  each  space  in  the  hierarchy  sorves  both  as  an 
abstraction  space  for  the  more  dotailed  space  just  bolow  it  and  as  a ground  space  with 
respect  to  the  less  detailed  space  Just  above. 


Background— The  STRIPS  System 

ABSTRIPS  is  a modification  of  the  STRIPS  systom,  described  in  Article  D5.  The  problem 
domain  for  both  programs  Is  a world  of  robot  planning.  In  both,  tho  program  Is  given  an  Initial 
state  of  tho  world,  or  world  model,  consisting  of  a sot  of  formulas  that  describe  the  floor  plan 
of  a group  of  rooms  and  other  facts  such  as  the  location  of  the  robot  and  other  objects 
within  those  rooms.  The  goal  state  to  be  achieved  is  also  given.  Tho  elements  of  a solution 
sequence  are  operators  representing  robot  actions;  examples  are  operators  for  going  up  to  an 
object,  pushing  an  object,  and  going  through  a door.  The  definition  of  each  operator  contains 
three  kinds  of  formulas;  (a)  its  preconditions,  representing  statements  that  must  be  true  of  a 
world  modol  In  order  for  tho  operator  to  bo  applicable;  (b)  its  add  list,  a list  of  formulas  that 
will  become  true  and  should  be  added  to  the  world  modol  when  the  operator  Is  applied;  and 
(c)  Its  delete  list,  a corresponding  list  of  formulas  to  bo  deleted  from  the  model  upon 
application  of  tho  operator.  Tho  search  for  a sequence  of  operators  producing  the  desired 
world  model  Is  guided  by  a means-ends  analysis  similar  to  that  of  GPS  (Article  02). 


Abstraction  Spaces 

Given  tho  world  models  and  operator  descriptions  of  tho  basic  STRIPS  system,  the  first 
question  Is  how  to  define  the  "details"  that  are  to  bo  ignored  in  the  first  pass  at  a solution. 
Sacerdoti's  answer  was  to  treat  as  dotails  certain  parts  of  tho  operator  preconditions.  At  all 
levels  of  abstraction,  tho  world  modols  and  the  add  and  delete  lists  of  operators  remain 
exactly  the  same.  Such  a definition  of  "details"  was  found  to  be  strong  enough  to  produce 
real  improvements  In  problem-solving  efficiency,  while  keeping  a desirable  simplicity  In  the 
relationship  between  each  abstraction  space  and  its  adjacent  ground  space. 

The  preconditions  for  an  operator  are  stated  as  a list  of  predications,  or  literals, 
concerning  the  world  model  to  which  the  oporator  is  to  be  applied.  The  relative  Importance 
of  literals  Is  Indicated  by  attaching  to  oach  a numbor  called  its  criticality  value.  The  hierarchy 
of  problem  spaces  is  than  defined  in  terms  of  levels  of  criticality:  In  the  space  of  criticality 
n,  all  operator  preconditions  with  criticality  less  than  n are  Ignored. 
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Tho  assignment  of  criticality  values  is  done  Just  once  for  a given  definition  of  the 
problem  domain.  The  general  ideas  to  be  refloctod  in  the  assignment  are  the  following: 

1.  If  the  truth  value  of  a literal  cannot  be  changed  by  any  operator  in  the 
problom  domain,  it  should  have  tho  highost  criticality. 

2.  It  tho  preconditions  for  an  operator  include  a literal  L that  can  be 
roadily  achioved  once  other  preconditions  for  the  same  operator  are 
satisfied,  thon  L should  bo  less  critical  than  those  other  preconditions. 

3.  If  tho  possibility  of  satisfying  literal  l depends  on  additional 
preconditions  besides  thoso  reforrod  to  In  (2),  then  L should  have  high 
but  loss  than  maximum  criticality. 

The  actual  assignment  of  criticalitios  is  done  by  a combination  of  manual  and  automatic 
moans.  First,  the  programmer  supplies  a partial  ordering  of  all  predicates  that  can  appear  in 
operator  preconditions.  Tho  partial  ordering  sorves  two  purposes:  It  supplies  a tentative 
criticality  value  for  all  Instances  of  each  prodicato.  and  It  govorns  the  order  in  which  the 
program  will  considor  literals  for  possible  increases  (but  not  decreases)  in  criticality. 

As  an  examplo,  considor  an  operator  TURN-ON-LAMP  (x),  with  preconditions 

TYPE  (x.LAMP)  and  THERE  EXISTS  (r)  [INROOM  (ROBOT, r)  and 

INROOM  (x,r))  and  PLUGGED-IN  (x)  and  NEXTTO  (ROBOT, x)]  . 

The  partial  ordering  of  predicates,  reflecting  an  intuitive  view  of  their  relative  importance, 
might  bo  as  follows: 


Predicate  Rank 

TYPE  4 
INROOM  3 
PLUGGED-IN  2 
NEXTTO  1 


Figure  1.  Initial  ranking  of  predicates. 

The  assignment  algorithm,  whoso  output  is  summarized  in  the  figure  below,  would  first  find 
that  tho  truth  of  TYPE  (x.LAMP)  is  beyond  tho  power  of  any  operator  to  change  and 
therefore  would  set  Its  criticality  to  the  maximum;  in  this  case,  6.  Then  it  would  find  that 
TYPE  (x.LAMP)  is  an  insufficient  basis  for  achieving  INROOM  (ROBOT, r)  or  INROOM  (x.r);  so 
these  two  literals  would  have  their  criticality  raised  to  the  next  highest  value,  5.  Next 
PLUGGED-IN  (x)  Is  considered,  and  a plan  to  achiovo  PLUGGED-IN  (x)  Is  found  using  only  the 
literals  already  processed  as  a starting  point.  Hence,  the  PLUGGED-IN  literal  retains  Its 
tentative  criticality  of  2,  and,  similarly.  NEXTTO  (ROBOT, x)  is  given  criticality  1.  The  result, 
after  similar  processing  of  the  preconditions  of  tho  other  operators  In  the  domain,  is  a 
hierarchy  of  at  least  four,  and  possibly  six,  distinct  problom  spaces. 
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Literal  Criticality  Value 

TYPE  (x. LAMP)  6 

INROOM  ROBOT, r)  5 

INROOM  (x,r)  5 

PLUGGEO-IN  (x)  2 

NEXTTO  ( ROBOT, x)  1 

Figure  2.  Final  criticality  of  literals. 

Control  Structure 

A problem  statement  for  ABSTRIPS,  as  for  STRIPS,  consists  of  a description  of  the  state 
of  the  world  to  be  achieved.  A solution  is  a plan,  or  sequence  of  operators,  for  achieving  it. 
ABSTRIPS  proceeds  by  forming  a crude  plan  at  the  highest  level  of  abstraction  and 
successively  refining  it.  The  executive  is  a recursive  program  taking  two  parameters:  the 
current  level  of  criticality,  dofining  the  abstraction  space  in  which  planning  is  to  take  place, 
and  a list  of  nodes  representing  the  plan  to  be  refined.  Before  the  initial  call,  criticality  is 
set  to  the  maximum,  and  tho  skeleton  plan  is  initialized  to  a single  operator--a  dummy-- 
whose  preconditions  are  precisely  the  goal  to  be  achieved.  ABSTRIPS  computes  the 
difference  between  tho  preconditions  and  the  current  world  model,  finds  operators  relevant 
to  reducing  the  difference  and,  if  necessary,  pursues  subgoals  of  satisfying  the 
preconditions  of  the  selected  operators.  During  this  process,  any  preconditions  of  less  than 
the  current  criticality  are  ignored.  A search  tree  is  built  from  which,  if  the  process 
succeeds,  a fuller  operator  sequonco  leading  from  the  initial  world  model  to  the  goal  can  be 
reconstructed.  This  new  skeleton  plan,  together  with  the  next  lower  criticality  level,  are 
passed  recursively  to  the  executive  for  the  next  round  of  planning. 

The  search  strategy  used  by  ABSTRIPS  can  be  called  Ungth-first,  in  that  the  executive 
| forms  a complete  plan  for  reactiing  the  goal  in  each  abstraction  space  before  considering 

plans  In  any  lower  level  space.  T his  approach  has  tho  advantage  that  it  permits  early 
recognition  of  dead  ends,  thus  reducing  tho  work  wasted  in  extending  the  search  tree  along 
fruitless  paths  involving  dotailed  preconditions.  If  a subproblcm  in  any  particular  space 
cannot  be  solved,  control  is  rcturnod  to  its  abstraction  space,  and  the  search  tree  is 
restored  to  its  previous  state  in  that  space.  The  node  that  caused  the  failure  in  the  lower 
level  space  is  eliminated  from  consideration  and  the  search  is  continued  in  the  higher  level 
space.  This  mechanism,  an  example  of  backtracking,  suffers  from  the  problem  that  no 
information  is  available  at  the  higher  level  on  what  caused  tho  plan  to  fail. 

« 

Because  backtracking  can  be  inefficient  and  also  because  each  operator  in  an 
abstraction  space  may  be  expanded  to  several  operators  in  the  ground  space,  it  is  important 
for  ABSTRIPS  to  produce  good  plans  at  tho  highest  level.  Two  modifications  to  STRIPS  were 
made  to  try  to  Insure  that  it  would  do  so. 

I First,  whoreas  a STRIPS  search  tended  to  be  depth-first  and  therefore  sometimes  found 

■ non-optimal  solutions,  ABSTRIPS  makes  tho  order  of  expanding  nodes  in  the  search  tree 

dependent  on  the  level  of  abstraction.  At  the  highest  level  It  uses  an  evaluation  function  that 

Lmay  Increase  the  soarch  effort  but  which  insures  that  the  shortest  possible  solution 
sequence  will  be  found.  (See  Article  C3b  on  A*.) 
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The  second  change  relates  to  tho  instantiation  of  operator  parameters,  in  cases  where 
two  or  more  choices  seem  equally  good.  While  STRIPS  made  a choice  arbitrarily,  ABSTRIPS 
defers  the  choice  until  a greater  level  of  detail  Indicates  one  to  be  preferable.  Backtracking 
can  still  occur  should  the  choice  be  mistaken. 


Performance 

ABSTRIPS  and  STRIPS  were  compared  on  a sequence  of  problems.  One  of  the  longest, 
needing  1 1 operators  for  its  solution,  required  the  robot  to  open  a door,  go  through  tho 
adjacent  room  to  another  room,  push  two  boxes  together,  and  then  go  through  two  more 
doors  to  reach  the  room  where  it  was  to  stop.  The  basic  STRIPS  system  required  over  thirty 
minutes  of  computer  time  to  find  the  solution;  ABSTRIPS  used  5:28  minutes  and  generated 
only  half  the  number  of  search-tree  nodes.  It  was  noted  that  by  the  time  ABSTRIPS  reached 
the  most  detailed  level,  It  had  in  effect  replaced  the  original  large  problem  by  a sequence  of 
7 easy  subproblems. 


References 


See  Sacerdoti  (1974). 
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